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FOREWORD 


AUERBACH  Corporation  was  awarded  a  contract  (Contract  Number 
SD-219)  by  the  Office  of  the  Secretary  of  Defense  to  perform  a  com¬ 
prehensive  survey  of  the  RDT&E  personnel  in  the  Department  of 
Defense  to  determine  how  these  individuals  acquire  and  utilize  tech¬ 
nical  and  scientific  information  in  the  conduct  of  specific  tasks 
associated  with  their  work. 

The  study  was  conducted  under  the  sponsorship  of  the  Advanced 
Research  Projects  Agency,  Office  of  the  Secretary  of  Defense;  ad¬ 
ministered  under  the  direction  of  Mr.  Fred  A.  Koethcr,  Director, 
Technical  Information,  Advanced  Research  Projects  Agency;  and 
directed  by  Mr.  Walter  M.  Carlson,  Director  of  Technical  Informa¬ 
tion,  Ofiice  of  the  Director  of  Defense  Research  and  Engineering, 
Department  of  Defense. 

In  addition  to  the  AUERBACH  staff  utilized  in  performing  this  study, 
outside  consultants  were  employed  in  specific  areas.  Dr.  John 
de  Cani,  of  the  Statistics  Department  of  the  University  of  Pennsylvania, 
provided  consultation  in  the  statistical  aspects  of  the  study;  Dr.  Herbert 
Menzel  from  Columbia  University  provided  Inputs  for  the  survey 
design;  and  Dr.  Robert  Sleight  and  Mr.  Kenneth  Cook  of  the  Applied 
Psychology  Corporation  were  consulted  in  the  use  of  interviewing 
techniques. 


The  report  is  presented  in  two  volumes.  Volume  I  consists  of  three 
sections  entitled;  Management  Report,  Conduct  of  Study,  and  Analysis 
ofDnta,  plus  four  appendices  entitled;  Study  Execution,  Discussion  and 
Results  of  Survey  Questions,  Bibliography  of  User  Studies,  and  Glossary 
Volume  11  consists  of  three  sections  entitled  Interview  Guide  Handbook, 
Computer  Program  Documentation,  and  Statistical  Tables. 


NOTE: 

The  detailed  da: a  collected,  supplementary  com}  alee  printouts, 
and  the  computer  programs  developed  io  compile  and  tabulate  the  data 
have  been  delivered  to  the  Department,  of  Defense.  Persons  interested 
in  these  items  should  contact: 

Director  of  Technical  Information 
Office  of  the  Director  of  Defense  Research 
and  Engineering 
The  Pentagon 
Washington,  U.C.  20301 
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SECTION  I.  MANAGEMENT  REPORT 


1.1  INTRODUCTION 

The  Importance  of  Scientific  and  Technical  Information 

The  acquisition,  use,  and  creation  of  scientific  and  technical  information 
is  an  integral  part  of  the  research,  development,  test,  and  evaluation  (RDT&E) 
process.  Information  is  an  essential  raw  material  and  generally  the  output  of  this 
process.  Within  the  Department  of  Defense  (DOD)  alone,  approximately  seven  billion 
dollars  were  spent  for  RDT&E  during  the  1964  fiscal  year.  Engaged  in  this  massive 
effort  are  some  36,000  scientists  and  engineers  within  the  Department  of  Defense 
and  an  even  larger  number  in  industry.  It  is  therefore  not  surprising  that  DOD,  the 
largest  sponsor  of  research  and  engineering,  should  be  paying  increasing  attention  to 
the  manner  in  which  scientific  and  technical  information  is  acquired,  processed, 
packaged,  delivered,  and  used  by  its  engineers  and  scientists. 

The  Information  Problem 

The  "information  problem"  is  generally  attributed  to  the  exponential  growth 
of  the  technical  literature.  In  reality,  there  are  a  series  of  information  problems 
affecting  different  groups  at  different  levels.  For  example,  the  librarian  and  docu- 
mentalist  are  faced  with  the  problem  of  managing  this  rapidly  growing  volume  of 
information.  Users  are  faced  with  the  problem  of  having  to  be  more  selective  in 
their  information  intake.  The  technical  man  no  longer  can  keep  up  with  all  the  litera¬ 
ture  in  his  field,  nor  even  read  all  of  the  material  which  may  be  relevant  to  a  particular 
task. 


Further,  each  time  a  new  technology  goes  through  a  period  of  rapid  growth, 
the  edges  of  disciplines  appear  to  become  more  blurred,  whole  new  areas  spring  forth, 
and  the  semantic  distortion  among  author,  systems  of  storage  and  retrieval,  and  user 
become  more  and  more  involved.  Under  such  conditions,  the  technical  man  is  fre¬ 
quently  unaware  of  prior  work  and  repeats  tasks  which  have  already  been  accomplished. 
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The  magnitude  and  importance  of  the  information  problem  have  resulted 
in  the  establishment  of  a  number  of  new  information  activities.  Most  of  these  are 
oriented  around  the  announcement  and  distribution  of  documents  —  usually  technical 
reports.  More  recently,  the  emphasis  has  shifted  to  technical  information  analysis 
centers  which  also  perform  the  functions  of  evaluation,  correlation,  and  synthesis 
of  information. 

It  is  not  at  all  clear,  however,  that  these  information  activities  are,  or 
will  ultimately  be,  effective  solutions  to  the  information  problem  since  the  needs  of 
the  ultimate  user  are  not  adequately  understood. 

The  present  situation  is  vividly  portrayed  by  Alan  Rees  of  Western 
Reserve  University  in  an.  article  in  Information  Retrieval  in  Action:^ 

"  ae  information  retrieval  field  has  been  plagued  for  many 
;  ire  by  busy  people  spending  large  sums  of  money,  de¬ 
signing  —  or  attempting  to  design  —  phantom  systems  for 
non-existent  people  in  hypothetical  situations  with  unknown 
needs.  It  is  not  surprising  that  large  numbers  of  theorists, 
hardware  peddlers  and  promoters  have  ignored  the  user, 
with  the  result  that  the  needs  of  users  are  conspicuously 
absent  in  many  discussions  on  system  design  and  operation.  " 

The  goal  of  future  information  systems  should  be  to  provide  the  right  infor¬ 
mation  to  the  right  person,  in  the  right  form,  at  the  right  time,  and  at  his  woi’k 
station.  To  achieve  this  goal,  it  is  first  necessary  to  define  the  user's  need  for  ac¬ 
quiring  and  utilizing  technical  information. 

Previous  User  Studies 

User  needs  are  neither  broadly  known  nor  well  understood,  despite  the 
fact  that  numerous  scientific  and  technical  use  studies  have  boon  conducted  and  re¬ 
ported  in  the  litorature.  The  Bibliography  included  in  Appendix  C  of  this  Volume  lists 
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Rees,  Alan  M. ,  "Information  Needs  and  Patterns  of  Usage."  Information 
Retrieval  in  Action.  Western  Reserve  University,  Cleveland  (1963)  pp.  17-23. 


nearly  700  titles  of  references  dealing  with  earlier  or  contemporary  user  studies. 
While  a  majority  of  these  studies  appear  to  have  been  well  conducted  and  have  pro¬ 
duced  valuable  information  for  specific  purposes,  they  do  not,  individually  or 
collectively,  provide  sufficient  general  criteria  for  defining  the  specific  information 
needs  of  any  broad  segment  of  the  DOD  RDT&E  community. 


There  are  a  number  of  reasons  for  this: 

(1)  A  large  number  of  the  studies  were  conducted  for 
specific  purposes  in  unique  environments;  e.  g. , 
studies  concerning  the  journal  reading  habits  of 
various  groups,  the  frequency  of  use  of  a  particular 
library,  the  readership  of  specific  publications  and 
the  media  or  channel  used  to  acquire  information. 

(2)  Many  of  them  have  been  concerned  only  with 
scientists  in  a  research  atmosphere,  usually  in  an 
experimental  laboratory  environment. 

(3)  Few  studies  have  been  concerned  with  engineers, 
particularly  those  with  a  product  or  administrative 
orientation. 

(4)  Most  ot  the  studies  have  concentrated  on  the  users' 
information  needs  in  terms  of  the  types  of  packages 
they  use,  e.  g, ,  journal,  rather  than  the  information 
they  need. 

(5)  Wide  differences  exist  in  the  sampling,  interviewing, 
and  data  analysis  techniques  used,  making  correlation 
of  results  difficult  or  impossible. 


Consequently,  few  of  the  studies  provide  a  reliable  base  for  extrapolation 
to  goneral  conclusions  about  the  information  needs  of  the  DOD  RDT&E  community. 


1.2  STATEMENT  OF  STUDY  OBJECTIVES 


The  primary  objective  of  the  DOD  user  need  study  was  to  collect  and  tuutly 
a  statistically  significant  data  base  on  how  DOD  scientists  and  engineers  presently 
acquire  and  utilize  technical  information  in  the  performance  of  their  tasks. 
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The  data  base  would  serve  to  answer  such  questions  as:  j 

•  Who  is  the  user  ? 

i 

l 

•  In  what  technical  activity  is  he  engaged?  1 

•  What  is  most  frequently  the  first  source  to  which  the 
DOD  technical  man  turns  for  technical  information? 

•  What  class  of  information  does  this  technical  man 
use? 

•  How  much  of  the  available  relevant  information  does 
he  want? 

•  Does  he  need  a  detailed  analysis  of  the  information? 

•  How  quickly  does  he  need  the  information? 

•  Does  he  have  difficulty  finding  information? 


Scope 

The  scope  of  the  study  is  defined  by  Article  I  of  the  contract  schedule  as 
quoted  below: 

"ARTICLE  I.  SCOPE.  Tne  Contractor  agrees  to 
conduct  a  study  relating  to  tM  availability  of  scientific  infor¬ 
mation  to  technical  personnel  involved  in  defense  activities, 
in  connection  therewith,  the  Contractor  will  perforin  the  fol¬ 
lowing  tasks: 

"1.  Establish  objectives,  procedures,  and  ground  rules 
under  which  the  study  is  t .  be  made  through  discussions  with 
the  personnel  of  the  Office  of  the  Director  of  Technical  Infor¬ 
mation,  Office  of  the  Direct^i  of  Defense  Research  and  Engineering. 

"2.  Analyze  the  j  apulation  under  study  using  tab  cards 
and  other  data  provided  by  the  Department  of  Defense,  and 
develop  sampling  procedures  and  design  the  sample  size. 

"3.  Develop  for  the  interview  survey,  necessary  guides, 
procedures,  materials,  and  administrative  controls,  and 
train  interviewers  in  cooperation  with  DOD  personnel,  if 
deemed  necessary. 
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"4.  Select  and  train  the  survey  team.  The  Department 
of  Defense  may,  at  their  option,  include  up  to  five  (5)  DOD 
personnel  in  the  training  program. 

"5.  Conduct  at  least  one  pilot  survey  of  the  fifty  (50)  or 
more  interviews  of  DOD  scientific  and  technical  personnel 
within  the  Washington,  D. C. ,  area.  Additional  pilot  surveys 
of  up  to  two  hundred  (200)  interviews  may  be  conducted,  with 
the  approval  of  the  contract  scientist,  as  required,  to  properly 
pi'epare  for  the  main  survey. 

"6.  Analyze  the  results  of  the  pilot  surveys,  revise  the 
interview  guide  and  procedures  as  required,  select  the  specific 
people  to  be  interviewed  according  to  the  sampling  procedure, 
and  make  final  preparations  for  the  main  survey. 

"7.  Conduct  a  survey  of  DOD  scientific  and  technical 
personnel  according  to  the  procedures  developed  in  the  above 
tasks.  Contractor  shall  conduct  not  less  than  twelve  hundred 
(1200)  interviews.  The  total  number  of  formal  interviews, 
including  pilot  and  main  study,  will  not  exceed  eighteen 
hundred  and  fifty  (1850). 

"8.  Under  the  direction  of  the  Director  of  Technical 
Information,  perform  the  preliminary  analysis  of  the  survey 
results  which  will  provide  the  basic  statistical  information 
and  a  summary  of  the  nonquantitative  results. " 


The  study  was  not  intended  to  develop  specifications  for  a  DOD-wide  infor¬ 
mation  system.  Such  a  task  would  necessarily  require  many  additional  types  of 
information  which  are  far  beyond  the  scope  of  this  effort.  It  was  intended,  however,  to 
be  a  first  step  toward  this  larger  goal,  as  it  forms  the  basis  for  further  analyses  and 
investigations  into  the  information  processes  and  needs  of  the  RDT&E  community 
within  the  Department  of  Defense. 


1.3  DATA  DASH  DEVELOPED 

A  scientifically  designed  survey  methodology  was  employed  to  gather  data. 

A  random  sample  of  1,375  DOD  KDT&E  personnel  was  selected  and  interviewed  by  the 
survey  team.  From  the  sample  data,  %  large  data  base  was  compiled.  This  data  is 
available  in  several  forms. 
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(1)  Approximately  600  selected  tables  which  combine 
answers  to  either  one,  two  or  three  questions  viewed 
together. 

(2)  The  completed  interview  guides,  as  submitted  by  the 
survey  team  interviewers. 

(3)  Coded  data  from  the  completed  interview  guides 
stored  on  standard  EAM  punched  cards. 

(4)  Coded  data  from  the  completed  interview  guides 
stored  on  magnetic  tape,  which  is  capable  of  being 
run  on  the  IBM  1401  computer.  Numerous  tables  of 
analyzed  data,  which  combine  answers  to  either  one, 
two  or  three  questions  viewed  together,  are  also 
available  on  magnetic  tape. 

(5)  Standard  EAM  punched  cards  containing  narrative 
responses  to  selected  questions. 


Various  computer  programs  were  prepared  and  used  for  statistically 
processing  and  formatting  the  data.  The  data  and  programs  are  available  for  further 
investigation  and  analyses. 


1.4  FINDINGS 

The  following  findings  represent  some  of  the  more  significant  statistical 
results  obtained  from  the  data.  For  ease  of  presentation,  some  of  the  data  categories 
have  been  pooled.  Detailed  breakdowns  of  the  data  are  presented  in  Section  III  and  in 
Appendix  B. 


Characteristics  of  the  Population 

Little  data  was  available  prior  to  t  his  study  about  the  characteristics  of  the 
DOD  RDT&E  population.  This  was  one  of  the  primary  reasons  for  using  a  random  rather 
than  a  stratified  sampling  technique.  The  use  of  a  random  sample  lias  produced  a 
comprehensive  picture  of  the  RDT&E  population. 

The  educational  background  of  the  DOD  HDT&E  pollution  is  heavily  oriented 
towards  the  engineering  sciences,  with  51  percent  of  ail  degrees  within  the  field  of 
engineering. 
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The  population  is  well  educated  as  90  percent  have  at  least  a  bachelor's 
degree  and  25  percent  have  an  advanced  degree.  The  educational  distribution  of  the 
population  is  shown  in  Figure  1-1. 


65% 

BACHELORS 


8% 
PH.  D. 


10% 

NO  DEGREE 


Figure  1-1.  Highest  Degree 


The  population  is  primarily  civilian  with  only  15  percent  being  military. 
The  median  age  was  found  to  lie  40  years. 

Consistent  with  the  educational  background,  about  half  of  the  job  classifi¬ 
cations  (MQS)  are  in  engineering  and  one  quarter  in  the  physical  sciences.  The  dis¬ 
tribution  of  technical  activities  in  which  the  DQD  RDT&E  population  is  engaged  was 
found  to  be  as  shown  in  Figure  1-2. 


DETAILED 
ENGINEERING 
OR  SCIENTIFIC 


29% 


TECHNICAL 
EVALUATION  j|% 

J  TECHNICAL 
ADMINISTRATION 


3% 

OTHER 


Figure  1  -2.  Type  of  Activity 
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Characteristics  of  the  Tasks 


The  DOD  technical  population  was  found  to  be  engaged  in  a  broad  spectrum 
of  technical  activities  in  all  scientific  fields,  with  the  major  emphasis  in  the  engineer¬ 
ing  sciences. 

Most  of  the  task  assignments  identified  involved  relatively  limited  effort 
(i.  e. ,  less  than  one  man-week);  the  more  extensive  taskj  were  generally  research- 
oriented.  The  technical  character  of  the  tasks  was  distributed  as  shown  in  Figure  1-3. 

50% 

DEVELOPMENT 

4% 


Figure  1-3.  Kind  of  Task 
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The  major  output  of  a  task  was  more  frequently  a  finding  or  recommenda¬ 
tion  rather  than  a  decision.  The  distribution  is  shown  in  Figure  i-4.  The  primary 
mode  of  presenting  task  results  was  in  writing  (68  percent),  as  compared  to  oral  or 
any  other  form. 


49% 

FINDING 


36% 
RECOMMENDATION 


18% 

DECISION 


4% 

NO  ANSWER 


Figure  1-4.  Major  Output  of  a  Task 
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Characteristics  of  the  Information  Acquired  and  Used 


The  major  emphasis  in  the  study  was  on  the  characteristics  of  the  infor 
mation  acquired  and  how  it  was  used.  The  more  important  findings  in  this  area 
are  as  follows: 


(1)  Classes  of  Information  Used.  The  most  frequently  used 
category  of  information  can  be  termed  ’’engineering 
data, "  particularly  the  class  entitled  performance  and 
characteristics  and  specifications  (42  percent). 

The  relative  usage  of  all  the  various  classes  oi  infor  ¬ 
mation  in  the  completion  of  the  sampled  tasks  is  shown 
in  Figure  1-5. 


PERFORMANCE  AND  CHARACTERISTICS 
SPECIFICATIONS 

DESIGN  TECHNIQUES,  EXPERIMENTAL 
PROCESSES,  PRODUCTION  PROCESSES 
AND  PROCEDURES,  AND  TEST 
PROCESSES  AND  PROCEDURES 

UTILIZATION 

TECHNICAL  STATUS 

CONCEPTS 

MATH  AIDS  AND  FORMULAE 
RAW  DATA 

COST  AND  FUNDING 
OTHER 


42% 


Figure  1-5.  Distribution  of  Classes  of  ltUorinaUon  Used 


The  distribution  of  information  by  class,  as  shown 
above,  appears  to  be  independent  of  either  the  ehar- 
aeterislies  of  the  task  or  the  characteristics  of  the 
person  interviewed.  That  is,  the  relative  distribution 
by  class  does  not  significantly  vary  with  the  person's 
MOS,  the  ty}K>  or  field  of  the  task,  the  length  of  task, 
or  the  task  output.  There  arc  a  few  minor  variations 
to  this  general  finding  in  the  fields  of  ordnance, 
med  cinc,  research,  and  aircraft,  which  are  discussed 
in  Section  lit. 
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(2)  Volume  of  Information  Ujed  (Recall).  How  much  of 
the  available  material  was  the  user  exposed  to  for  a 
given  task  related  information  requirement?  Only  oc 
casionally  (18  percent)  did  he  see  (or  believe  he  saw) 
all  of  the  available  material  (see  Figure  1-6). 


36% 

ONE  ITEM  OF  34  % 


Figure  1-6.  Volume  of  Information  Used 


(3)  Search  Aids.  Would  tfte  user  have  found  title 

listings  or  abstracts  useful  to  read  first  in  order 
to  select  chunk*  material  to  read  in  detail? 

This  question  was  asked  only  if  the  interviewee 
indicated  a  need  for  a  large  volume  of  information 
on  the  particular  chunk.  The  specific  responses 
to  this  question  were  distributed  as  shown  in 
Figure  1-7, 


53% 

SEARCH  AIDS 
WOULD  BE  31% 

USEFUL  SEARCH  AIDS 


Figure  1-7.  Utility  of  Search  Aids 


*  A  chunk  is  the  smallest  unit  of  task  required  information 
which  would  lose  its  identification  and  meaning  with  re¬ 
spect  to  the  task  if  segmented  further. 
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(4)  Retrieval  Time.  Information  was  generally  obtained  in 
less  than  the  allowable  time,  as  shown  in  Figure  1-8. 
For  example,  in  only  18  percent  of  the  cases  was  Jie 
information  needed  in  less  than  one  day,  whereas  it  was 
actually  obtained  in  less  than  one  day  in  41  percent  of 
the  cases. 


PERCFNT 

CF 

CASES 


□  PERCENTAGE  OF  CASES  IN  WHICH 
INFORMATION  WAS  OBTAINED  IN 
LESS  THAN  THE  ALLOWABLE  TIME. 


Figure  1-8.  Retrieval  lime 


These  distributions  appeared  to  lie  independent  of  the 
media  in  which  the  information  was  obtained  or  desired. 
The  time  requirement  observed  may  net  be  toe  signifi¬ 
cant  because  people  tend  to  adjust  their  time  require¬ 
ments  for  information  to  the  existing  system. 


(5)  First  Source  of  Information.  In  more  than  half  of 
the  task  related  searches,  RDT&E  personnel  utilized 
colleagues,  personal  files,  and  local  departmental 
sources  as  their  first  source  of  information  as  illus¬ 
trated  in  Figure  1-9.  Furthermore,  regardless  of  the 
first  source  contacted,  this  source  completely  satisfied 
tiie  particular  information  requirement  39  percent  of 
the  time.  Thus,  a  convenient  and  responsive  external 
first  source  is  an  important  element  of  the  RDT&E 
information  system. 


LIBRARY  and 
INFORMATION  CENTERS 
(5%) 

MANUFACTURER 
or  SUPPLIER 
(5%) 

CONSULTANTS 
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SUPERVISOR 
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ASSIGNED  TO  SUBORDINATE 
(4%) 


Figure  1-0,  First  Source  of  Task  Related  Information 


(6)  Media.  Oral  communication  was  found  to  be  a 

significant  vehicle  for  acquiring  task  related  infor¬ 
mation.  Nearly  :*«  percent  of  all  su^h  information 
was  acquired  by  this  medium.  This  is  consistent 
with  the  findings  that  a  colleague  is  an  important 
first  source.  The  technical  report,  which  provided 
If*  jmreent  of  the  information  requirements,  was 
the  most  frequently  used  written  medium. 


(7)  Depth  of  Information.  Depth  was  defined  as  either  a 
"once  over  lightly, "  such  as  a  bibliography  which 
provides  indicative  abstracts  of  possibly  relevant 
documents,  or  a  "detailed  analysis, "  such  as  a  com¬ 
prehensive  state-of-the-art  review  of  a  technical 
field.  A  "specific  answer"  was  defined  as  a  sub¬ 
category  of  "detailed  analysis. " 

It  was  found  that  the  user  far  more  frequently  required 
a  "detailed  analysis"  or  a  specific  fact  than  a  "once 
over  lightly"  of  a  subject.  Figure  1-10  shows  the 
distribution  for  the  depth  of  information  received 
and  wanted. 
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1% 
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DEPTH  OF  INFORMATION  RECEIVED 
DEPTH  OF  INFORMATION  WANTED 

Figure  1-10.  Depth  of  Information 
Use  of  POD  Information  Services 

Several  questions  were  directed  specifically  to  the  use  of  POD  information 
services.  It  was  found  tluit  these  services  were  used  by  only  about  half  of  the  popula¬ 
tion,  and  were  rarely  relied  on  as  a  first  sources  of  information.  The  RDT&K  per¬ 
sonnel  who  utilised  POP  information  centers  were  generally  satisfied  with  the  serviu 
they  received.  A  considerable  proportion  (21  percent)  of  1UVF&E  jsersomiel  were  un¬ 
aware  of  the  existence  of  the  Defense  Documentation  Center  (DDC).  A  similar  propor¬ 
tion  (10  percent)  were  unaware  of  the  existence  of  any  one  of  33  specRied  D0D 
specialised  information  centers.  If  was  observed,  however,  that  many  users  may 
liave  actually  uulitcil  tin*  services  provided  by  tltese  organisations  without  being  aware 
that  they  were  doing  so.  Local  l  Un  a  l  ies  secure  services  for  a  user  f  rom  DDC  or 
DOD  informalitm  analysis  centers  and  sometimes  circulate  their  current  awareness 
materials,  such  as  DDC's  Technical  Abstract  Rultelm  (TAR),  in  oUier  than  their 
original  form.  As  a  result,  the  user  may  uoi  know  that  this  information  came  from 
DDC. 
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Satisfaction  With  Ability  to  Obtain  Information 


The  majority  (67  percent)  of  DOD  RDT&E  personnel  stated  that  they  had 
no  serious  trouble  obtaining  or  locating  information  needed  to  perform  or  conclude 
their  (asks  during  the  last  year. 

It  is  significant,  however,  that  as  many  as  27  percent  of  the  population 
s  eated  that  they  did  have  problems  in  obtaining  or  locating  information.  One  com¬ 
monly  encountered  suggestion,  by  the  respondents  who  did  claim  to  have  information 
problems,  was  that  DOD  publish  and  distribute  a  directory  or  register  of  the  R&D 
effort  currently  under  way  throughout  DOD. 

1.5  CONCLUSIONS 

The  above  findings  represent  the  most  significant  statistical  results  of  the 
study.  They  have  been  presented  essentially  without  comment  or  interpretation. 

The  conclusions  which  follow  attempt  to  interpret  both  the  moaning  of  some  o*  these 
results  and  their  significance  to  DOD. 

The  reader  may  observe  that  many  of  the  findings  and  conclusions  drawn 
from  this  study  are  in  agreement  with  widely  held  intuitive  opinions  of  many  knowl¬ 
edgeable  individuals  in  the  information  field.  The  data  developed  in  this  study  confirms 
and  answers  once  and  for  all  many  of  these  opinions,  and,  therefore,  takes  them  out 
of  the  realm  of  conjecture. 

importance  nf  Engineering  Data 

The  universal  use  of  engineering  date  throughout  the  KDT&K  community  is 
a  measure  of  its  importance  to  the  UDT&£  effort.  It  is  particularly  noteworthy  that 
forty -two  jsereent  of  the  users  repaired  specifications  and  performance  auu  character¬ 
istics  data  in  sujipoM  of  their  tasks.  Continued  improvement  of  the  methods  i:..* 
pivjia  ration,  Acquisition,  classification,  indexing,  storage  dissemination,  retrieval, 
and  correlation  of  such  engineering  data  is  clearly  warranted. 
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Significance  of  Local  Environment 


The  user  tended  to  rely  heavily  on  his  local  environment  (51  percent)  as 
a  first  source  of  information,  which  completely  satisfied  his  particular  information 
requirement  in  39  percent  of  the  cases.  This  finding  tends  to  confirm  the  existence 
and  significance  of  an  informal  information  system  consisting  of  the  user’s  personal  files, 
his  colleagues,  and  other  local  sources  of  information.  The  features  of  this  informal 
information  system,  which  the  user  apparently  considers  important,  are:  convenience, 
responsiveness,  and  the  ability  to  conduct  a  dialogue  (interplay  and  feedback).  The 
user  apparently  wants  to  deal  with  a  system  wherein  he  can  p  asonally  explain,  clarify, 
and  modify  his  requirements,  and  he  can  expect  in  response  the  right  information  in 
the  right  amount,  in  the  right  form,  and  in  the  time  required. 

The  fact  that  local  sources  of  information  are  heavily  utilized  may  also  in¬ 
dicate  that  DOD  is  well  managed  in  that  it  properly  assigns  tasks  to  that  organization 
which  has  adequate  background  and  experience  to  achieve  the  desired  technical 
objectives. 

Importance  of  the  Information  Analysis  Function 

The  high  proportion  (68  percent)  of  requirements  for  either  a  "detailed 
analysis"  or  "specific  answer, "  rather  than  a  "once  over  lightly"  (15  percent),  under¬ 
scores  the  importance  of  the  information  analysis  function.  The  user  apparently  .'.ants 
a  detailed  analysis  or  specific  answer  more  often  than  a  series  of  abstracts  to  docu¬ 
ments  which  may  be  relevant  to  his  question.  The  recent  emphasis  on  the  establish¬ 
ment  of  additional  formal  DOD  information  analysis  centers,  as  evidenced  by  DOD 
instruction  5100.45  entitled  Centers  for  Analysis  of  Scientific  and  Technical  Information, 
indicates  a  .  ocoguition  of  the  importance  of  this  function. 

Use  oi  the  V; _ . -mat  Information  System 

It  was  found  that  the  components  of  the  DOI)  formal  information  system  (tech¬ 
nical  libraries,  DOD  information  analysis  centers,  and  Defense  Documentation  Centers) 
were  not  widely  utilized.  One  reason  found  for  the  lack  of  widespread  use  was  a  lack 
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(4)  A  serious  information  problem  may  exist  with 
certain  types  of  information,  but  not  with  others. 

For  example,  one  area  identified  by  those  who 
claimed  to  have  a  problem  was  the  need  for  in¬ 
formation  on  current  R&D  projects  in  progress. 

1.(5  R  EC  OMMEls  PATIONS 

The  findings  and  conclusions,  which  have  been  developed  from  an  analysis 
of  the  data,  provide  the  basis  for  a  number  of  recommendations.  Since  the  study 
objective  was  to  develop  a  broad  data  base  on  information  use  patterns  within  the 
DOD  R13T&E  community,  and  not  a  blueprint  for  DOD-wide  information  systems,  it 
is  clear  that  no  drastic  or  precipitous  actions  are  warranted  directly  as  a  result  of 
this  study.  Much  more  still  needs  to  be  learned  about  the  user  as  well  as  about  DOD 
systems. 


The  recommendations  primarily  call  for  the  conduct  of  specific  studies 
and  experiments  in  those  areas  identified  as  significant  by  the  study.  The  experi¬ 
ment?  should  be  conducted  both  to  study  the  user  in  greater  detail  and  to  determine 
the  feasibility  of  specific  system  improvements,  their  potential  payoff,  the  problems 
encountered,  and  the  details  of  their  solution.  These  experiments  should  be  set  up 
at  a  small  number  of  installations  in  a  well -controlled  environment  to  assess  their 
effectiveness,  as  well  as  to  gain  a  more  complete  understanding  of  the  user  and  his 
information  requirements. 

Improvements  to  the  Local  Environment 

The  study  demonstrates  the  importance  of  the  local  environment  and  con¬ 
firms  the  existence  of  an  informal  information  system.  The  following  recommenda¬ 
tions  would  tend  to  strengthen  and  perpetuate  the  informal  information  system,  and,  if 
properly  implemented,  could  also  strengthen  the  formal  information  system.  Recom¬ 
mendations  concerning  the  interface  relationships  between  the  formal  and  informal 
information  systems  are  given  on  page  1-21. 


Preceding  Page  Blank 
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(X)  Local  Skills  Inventory.  Since  the  user  prefers  to 
ask  a  knowledgeable  colleague  for  information,  it 
may  be  useful  to  expand  the  number  of  colleagues 
he  has  access  to  by  providing  each  user  with  a  skills 
directory  of  the  personnel  at  his  particular  installation. 

An  experimental  directory  should  be  developed  which 
would  be  organized  by  skill  area,  and  would  list  such 
items  as  the  individual's  name,  telephone  number,  and 
experience  in  the  subject  area.  The  directory  would 
be  distributed  to  all  RDT&E  personnel  at  an  installa¬ 
tion.  Encouragement  would  be  given  to  individuals  to 
contact  the  listed  personnel  when  information  in  a  par¬ 
ticular  skill  area  is  required. 

Such  a  directory  would  serve  to  extend  the  concept  of 
a  knowledgeable,  available  colleague  who  can  provide 
rapid  feedback.  It  would  be  relatively  simple  to  de¬ 
velop  and  maintain  and  yet  might  make  a  significant 
impact  upon  the  ability  of  RDT&E  personnel  to  acquire 
and  utilize  information.  If  the  experiment  proves  suc¬ 
cessful,  this  concept  could  be  extended  throughout 
DOD  on  a  local  basis.  Ultimately,  the  local  skills 
inventories  could  be  integrated  into  a  central  DOD -wide 
skills  inventory  system.  Such  an  inventory  would  be 
useful  to  management  for  making  proper  task  assign¬ 
ments  at  the  DOD  level  as  well  as  at  the  agency  and 
local  levels.  This  inventory  need  not  be  widely  dis¬ 
tributed.  However,  a  user  could  make  inquiry  of  the 
central  system  where  he  needed  other  than  local  skills 
information. 

(2)  Improved  Personal  Files.  RDT&E  (iersonnel  and  de¬ 
partmental  organizations  should  be  encouraged  to 
organize  and  maintain  their  own  files  in  a  more  effec¬ 
tive  manner.  This  would  tend  to  decrease  the  access 
time  to  desired  information. 

An  experimental  training  course  or  guide  might  be 
developed  to  bring  to  Urn  attention  of  RDT&E  personnel 
techniques  currently  available  to  assist  them  in  better 
organizing  their  persona!  files.  Improved  filing 
systems  and  filing  aids  should  lx>  develojwd  and  tested 
on  an  experimental  basis.  Consideration  might  further 
be  given  to  determining  the  interrelationship  among 
personal  files,  departmental  files,  and  information 
systems  since  there  is  a  degree  of  overlap  betwf  un  the 
classification  and  indexing  problems  at  each  of  these 
points. 
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(3)  Information  Specialists.  It  was  observed  that  an 

information  specialist  was  being  utilized, in  a  number 
of  DOD  RDT&E  installations,  to  both  anticipate  and 
fulfill  the  information  requirements  of  the  technical 
staff.  The  information  specialist  frequently  acts  as 
an  intermediary  between  the  user  and  the  formal 
information  system.  It  is  recommended  that  an  ex¬ 
perimental  study  be  conducted  to  evaluate  the  utility 
of  an  information  specialist,  particularly  in  relation 
to  his  function  of  creating  new  information  (by  infor¬ 
mation  synthesis  and  correlation),  and  to  his  respons¬ 
ibility  for  monitoring  and  improving  the  user's  process 
of  information  acquisition,  use,  and  generation. 


Improvements  to  the  Formal  Information  System 

(1)  Publicity  and  Training.  To  increase  the  utilization  of  the 
DOD  formal  information  system,  the  technical  com¬ 
munity  should  be  made  aware  of  the  various  services 
available.  This  is  necessary  even  if  the  community 
usually  obtains  these  services  through  an  intermediary, 
such  as  an  information  specialist  or  a  local  library. 
Consequently,  an  experimental  public  relations  program 
is  recommended  to  promote  selected  DOD  information 
services. 


An  experimental  training  program  should  also  be  pre¬ 
pared  which  would  serve  to  provide  general  instruction 
in  the  use  of  various  DOD  information  centers  and 
services.  A  more  comprehensive  training  program 
and  handbook  should  be  prepared  (for  librarians  and  in¬ 
formation  specialists)  on  the  details  of  how  to  use  the 
various  formal  DOD  information  services.  This  would 
supplement  and  reinforce  the  more  general  training 
program  for  the  technical  man. 

(2)  Engineering  Data  Systems.  The  findings  and  conclusions 
suggest  that  greater  emphasis  should  be  given  to  systems 
for  handling  engineering  data,  such  as  performance  and 
characteristics  and  specifications.  Considerably  more 
must  he  known  about  tin*  engineering  data  requirements 
of  the  user  before  effective  engineering  data  systems  can 
be  designed. 
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It  is  therefore  recommended  that  further  DOD  user 
studies  be  conducted  which  focus  more  specifically  on 
engineering  data  requirements,  particularly  on  the 
specific  classes  and  sub-classes  of  engineering  data, 
and  the  degree  of  analysis  and  specificity  required. 


It  is  also  recon  mended  that  a  thorough  systems 
analysis  and  evaluation  be  made  of  existing  DOD  en¬ 
gineering  data  services  to  determine  whether  any 
significant  improvements  can  be  made  at  nominal 
cost. 


Further,  it  is  recommended  that  the  computer  mechan¬ 
ization  possibilities  of  an  engineering  data  system  be 
investigated.  Careful  consideration  should  be  given  to 
developing  means  for  easy  access  to  mechanized  engineer¬ 
ing  data  systems.  For  example,  the  utility  of  a  flexible 
on-line  query  system  might  be  studied.  With  a  flexible 
on-line  query  system,  the  user  could  interact  more  ef¬ 
fectively  with  a  mechanized  information  system,  which, 
in  essence,  would  be  acting  like  a  very  knowledgeable 
colleague. 


(3)  Utility  of  Search  Aids.  An  experiment  could  be  con¬ 
ducted  to  determine  if  increased  use  of  abstract  journals 
indexes,  title  lists,  and  similar  aids  by  the  technical 
man  would  materially  aid  him  in  selecting  and  acquiring 
technical  information.  A  training  program  on  the  avail¬ 
able  aids  and  their  use  could  be  developed.  This  would 
bo  accompanied  by  supplying  the  installation  with  appro¬ 
priate  current  awareness  and  search  aids,  and,  at  the 
same  time,  providing  the  necessary  support  capability  to 
allow  the  users  to  obtain  quickly  the  required  material. 

At  the  end  of  a  given  period  of  time,  the  use  and  applica¬ 
tion  of  current  awareness  and  search  aids  could  Ixs  exam¬ 
ined  to  obtain  a  qualitative  measure  of  their  utility. 


(•1)  U&D  Projects  Directory.  It  is  recommended  that  the 
development  of  the  DOD  K&l)  project  reporting  system 
(DOD  instruction  7720. 13)  be  accelerated  and  enhanced. 
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Suggestions  to  this  effect  were  received  from  a  number 
of  the  RDT&E  people  interviewed.  In  particular,  it  is 
recommended  that  a  directory  of  R&D  projects  in 
progress,  arranged  by  subject-category,  be  published 
and  widely  distributed  throughout  DOD,  within  security 
limitations.  Separate  cumulative  editions  might  be 
prepared  for  each  subject-category.  The  system  de¬ 
velopment  should  be  coordinated  with  other  groups  in¬ 
volved  in  this  problem,  such  as  the  Federal  Clearinghouse 
for  Scientific  and  Technical  Information,  which  plans  to 
publish  a  government  wide  directory  of  R&D  projects  in 
progress,  and  the  Science  Information  Exchange. 

A  major  difficulty  with  such  a  system  is  obtaining  com¬ 
plete  and  accurate  disclosure  of  R&D  projects.  To  avoid 
the  problems  associated  with  inaccurate  disclosure,  the 
system  must  provide  for  validating  the  reported  informa¬ 
tion,  perhaps  by  reference  to  collateral  documents  such 
as  technical  reports,  proposals,  and  the  like. 


Relationship  Between  Formal  and  Informal  Information  Systems 

It  is  recommended  that  a  study  be  conducted  to  define  more  clearly  the 
relationship  which  may  exist  between  the  formal  and  informal  information  systems. 

For  example,  DOD  current  awareness  services,  including  announcement  journals  and 
initial  and  secondary  distribution  of  technical  reports,  contribute  to  the  utility  of  the 
local  environment.  In  addition,  there  is  a  degree  of  overlap  between  the  indexing  of  a 
report  by  an  information  center  and  the  filing  of  that  same  report  in  a  personal  or 
departmental  file. 

The  study  should  determine  what  funutions  are  presently  performed  by  the 
informal  system  and  what  functions  are  performed  by  the  formal  system.  The  areas 
of  overlap  should  be  isolated.  The  degree  of  dependence  of  the  informal  on  the  formal 
system  should  be  identified  and  the  points  of  interface  clearly  defined.  Finally,  the 
major  role  of  each  component  of  the  overall  information  system  should  Ik?  determined. 
For  example,  one  component  may  produce  information,  another  may  be  the  wholesaler, 
while  still  another  may  act  as  the  retailer.  The  overall  system  also  includes  pro¬ 
ducers,  wholesalers,  and  retailers  of  specialty  products  and  services.  The  products 
and  services  at  each  level  of  distribution  should  be  identified,  analysed,  and  pre¬ 
cisely  described. 
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Experimental  Information  System 

It  is  recommended  that  an  experimental  information  system  be  set  up  to 
simulate  the  features  of  various  types  of  information  systems,  including  the  spec¬ 
ialized  information  center,  the  documentation  center,  the  local  library,  the  infor¬ 
mation  specialist,  and  others. 

The  primary  objectives  of  such  an  experimental  system  would  be  to 
observe  user  habits  and  to  accumulate  detailed  data  on  such  items  as:  (1)  traffic 
volume,  (2)  response  time,  (3)  vocabulary,  (4)  type  and  complexity  of  questions, 

(5)  the  need  for  information  or  data,  and  (6)  type  of  task  on  which  information  is 
required. 

A  further  objective  of  this  experiment  would  be  to  determine  the  type  and 
degree  of  mechanization  that  might  practically  be  effected  to  support  the  user. 

Some  of  the  above  objectives  are  difficult  to  accomplish,  and  can  only  be 
achieved  by  people  who  are  in  direct  communication  with  the  user.  Consequently, 
the  systems  would  rely  heavily  upon  people  to  perform  the  services  desired  by  the 
user,  including  many  services  which  might  more  appropriately  be  mechanized.  A 
sufficient  number  of  people  should  be  made  available  to  respond  immediately  to  user 
requirements.  The  people  operating  the  experimental  system  would  also  provide 
the  interface  with  other  information  centers,  which  may  be  unknown  to  the  user. 

The  informal  information  system  apparently  has  the  characteristics 
(convenience,  responsiveness,  and  the  ability  to  conduct  a  dialogue)  desired  by  the 
user.  A  reasonable  question  to  ask  is  whether  an  enhanced  formal  information 
service  which  had  these  characteristics  would  be  useful  and  utilized  extensively.  To 
develop  a  comprehensive  system  employing  the  aforementioned  characteristics  for 
the  entire  DOD  RDT&E  population  would  require  a  major  capital  expenditure.  It  is 
therefore  necessary  that  the  utility  of  such  a  system  be  determined,  to  the  degree 
passible,  without  a  full-scale  implementation. 
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Measurement  of  Information 

From  the  outset  of  this  study,  it  was  recognized  that  one  of  the  most 
severe  handicaps  was  the  unavailability  of  objective  and  precise  techniques  for 
measuring  information.  Measures  are  needed  for  describing  information  value, 
depth,  relevance,  recall  or  volume,  specificity,  and  packaging.  This  problem 
is  not  unique  to  this  study;  rather  it  is  a  problem  which  is  impeding  the  state  of 
the  art  of  the  entire  field  of  information  storage  and  retrieval.  Until  measurement 
techniques  are  developed,  future  studies  that  examine  non-quantifiable  information 
will  continue  to  be  hampered.  This  study  developed  and  used  the  concept  of  a 
"chunk  of  information"  as  a  subjective  measure  of  information.  This  concept, 
although  not  an  objective  or  precise  measure,  was  found  to  be  quite  useful  since 
no  existing  measures  seemed  to  serve  the  purposes  of  this  study.  Because  of  the 
urgency  of  this  problem  and  the  paucity  of  knowledge  on  this  subject,  we  recommend 
that  DOD  support  basic  research  work  toward  developing  useful  techniques  for 
measuring  information. 

1.7  STUDY  LIMITATIONS 

In  accordance  with  the  study  objectives,  a  broad  data  base  was  compiled 
and  an  analysis  of  the  data  made.  The  data  base  will  provide  the  basis  for  further 
analysis,  including  the  testing  of  new  hypotheses  which  stem  from  a  study  of  this 
report. 

In  making  further  analyses  of  the  survey  data,  the  reader  should  be 
aware  of  the  study  limitations.  Tim  study  was  broad  in  scope  and  dealt  with  a 
heterogeneous  inter-disciplinary  population  engaged  in  a  wide  variety  of  tasks.  How¬ 
ever,  the  sample  size  was  sufficiently  large  that  meaningful  subsets  of  the  population 
were  able  to  be  examined.  Because  of  the  breadth  of  the  study,  it  was  not  possible 
to  examine  each  specific  subset  of  the  population  to  a  maximum  degree  of  depth.  Tim 
scope  of  the  study,  however,  has  enabled  us  to  develop  a  comprehensive  picture  of 
I  the  information  patterns  of  the  entire  DOD  HDT&E  community. 

I 
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Considerable  care  was  taken  to  minimize  bias  in  the  data.  To  this  end, 
a  moderately  structured  interview  guide  and  interview  handbook  were  prepared  and 
utilized  to  increase  the  consistency  of  question  interpretation  and  the  resultant 
responses.  Interviewers  with  scientific  and  technical  training  and  experience  were 
selected  and  thoroughly  trained  in  the  purpose  of  the  survey  and  in  the  interviewing 
techniques  employed.  The  sample  size  was  sufficiently  large  that  the  sample  char¬ 
acteristics  will  differ  from  those  of  the  population  by  less  than  2. 7  percent,  in  95 
out  of  100  cases.  Standard  statistical  tests  such  as  the  Chi-Squared  test  were  applied 
where  appropriate .  A  critical  incident  interviewing  technique  was  employed  to 
identify  the  information  needed  or  used  for  a  recent  specifically  defined  task,  rather 
than  rely  on  general  opinions  and  recollection  of  distant  happenings. 

In  spite  of  the  above  precautions,  there  are  still  a  number  of  limitations 
to  the  data  and  possibilities  of  bias.  Several  of  the  questions  produced  marginal 
results  because  of  their  subjective  nature,  or  because  the  answer  categories  did  not 
pro'  ?  to  be  mutually  exclusive.  These  limitations  are  noted  in  Anpendix  B  in  the 
discussion  of  the  responses  to  each  question.  The  scientist  or  engineer  may  be  sub¬ 
consciously  biased  against  formal  information  systems  because  he  may  fear  losing 
liis  intellectual  freedom,  or  having  to  "pay”  for  such  formal  systems  by  a  corres¬ 
ponding  decrease  in  his  budget. 

The  study  did  not  cover  the  user’s  current  awareness  patterns  to  any  signi¬ 
ficant  degree.  During  the  course  of  the  interview  guide  development,  it  was  deter¬ 
mined  tint  current  awareness  questions  could  not  be  properly  handled  because  the  type 
of  questions  required  would  be  inconsistent  with  the  critical  incident  technique.  It  was 
also  felt  tliat  personal  interviewing  was  not  the  most  effective  means  for  gaining  insight 
into  current  awareness  patterns.  Close-up  observation  and  monitoring  of  the  informa¬ 
tion  activities  of  a  segment  of  the  HDT&E  imputation  may  be  the  only  reliable  way  to 
obtain  such  insight.  This  problem  was  brought  to  the  attention  of  the  DGD  and  it  was 
decided  that  specific  questions  on  current  awareness  would  be  omitted  from  the  study. 

U  should  be  suited  here,  however,  that  current  awareness  needs  are  important.  Con¬ 
sequently,  the  DOD  should  direct  itself  to  the  problem  of  developing  appropriate  methods 
and  surveys  to  determine  the  current  awareness  patterns  and  needs  of  the  technical 
community. 
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This  study  represents  a  significant  step  towards  a  better  understanding 
of  the  user  requirements  for  scientific  and  technical  information.  More  must  be 
learned  before  we  will  be  able  to  design  responsive  information  systems  which 
adequately  meet  the  actual  needs  of  the  user.  We  should  study  in  more  detail  the 
specific  requirements  of  particular  segments  of  the  DOD  population  for  engineering 
data,  because  of  its  relative  importance  to  the  entire  RDT&E  effort.  We  must 
study  the  relationships  between  the  informal  and  formal  information  systems  to 
determine  where  and  how  the  various  information  processing  functions  are  best 
performed.  Finally,  we  must  determine  how  current  awareness  information  is  ob¬ 
tained  and  the  relationship  between  current  awareness  and  retrospective  searching 
of  local  or  personal  files. 


SECTION  II.  CONDUCT  OF  STUDY 


2. 1  INTRODUCTION 

This  Section  describes  the  conduct  of  the  study  in  sufficient  depth  to  provide 
the  reader  with  the  background  information  necessary  to  interpret  properly  the  various 
analyses  of  the  data  presented  in  Section  III  and  Appendix  B  of  Volume  I,  and  in  Section  VI 
of  Volume  II.  A  more  detailed  discussion  of  the  study  execution  and  the  methodology 
employed  is  presented  in  Appendix  A  of  this  volume. 

The  various  steps  in  executing  the  study  are  shown  in  Figure  2-1.,  The  main 
path  of  the  study  is  indicated  by  the  heavy  arrow.  Descriptions  of  the  various  stages  are 
presented  in  the  subsequent  paragraphs. 

2 . 2  DEVELOP  STUDY  PLAN 

The  first  step  in  the  study  process  was  to  develop  an  overall  plan.  The  planning 
effort  resulted  in  initial  decisions  which  ruled  out  certain  types  of  survey  methods  such  as 
mail  questionnaires,  telephone  questionnaires,  ora  diary  method. 

Criteria  and  procedures  were  established  for  selecting  and  training  interviewers. 
Philosophies  were  also  developed  covering  sampling  methods,  interview  techniques  and 
a  method  for  testing  them,  survey  administration,  data  compilation,  and  methods  of  analysis 

2 •  REVIEW  PREVIOUS  USER  NEED  STUDIES 

Early  in  the  study,  a  comprehensive  bibliography  of  previous  and  contemporary 
user  need  studies  was  developed.  The  references  were  reviewed  for  ideas  or  techniques 
which  might  be  applicable  to  the  1X)D  User  Need  Study.  The  primary  value  of  this  review 
was  identifying  the  problems  which  may  arise  and  the  limitations  of  most  of  the  other 
studies.  A  bibliography,  which  includes  a  categorization  of  these  studies,  is  presented 
in  Appendix  C  of  this  volume. 

It  was  fouud  that  many  earlier  user  studies  suffered  from  the  weakness  of  being 
based  on  the  opinion  of  the  user.  The  problem  is  that  the  user  may  not  have  an  adequate 
basis  for  rendering  a  reliable  opinion  as  to  the  types  or  ctiaracierislies  of  information 
services,  since  he  may  not  have  liad  any  direct  experience  with  them. 
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It  was  therefore  decided  at  the  outset  to  make  sure  that  the  study  would  not 
become  an  opinion  survey.  This  objective  was  accomplished  by  directing  most  of  the 
questions  to  the  subject  of  what  information  was  actually  used  to  perform  a  recently- 
completed  task.  Relatively  few  questions  were  concerned  with  defining  the  kind  of  infor¬ 
mation,  product,  or  sc  twice  (lie  user  might  like  to  have. 

2 • 4  DEVELOP  SURVEY  METHODS 

A  key  feature  of  the  survey  methodology  was  the  application  of  the  critical 
incident  interviewing  technique.  This  technique  provides  that  a  specific  incident  be 
isolated,  in  this  case  a  recently-completed  task,  and,  through  a  series  of  questions,  the 
attributes  and  characteristics  of  the  information  actuary  utilized  in  the  performance  of 
this  particular  task  are  defined.  I'o  provide  a  basis  for  analysis  of  the  details  of  the 
critical  incident,  other  questions  are  posed  to  gain  insight  into  the  respondent's  background 
a iul  some  of  his  general  information  gathering  habits. 

2.5  INTERVIEW  GUIDE  AND  HANDBOOK 

A  semi-structured  interview  guide  was  developed  to  implement  the  critical  inci¬ 
dent  technique.  It  is  important  to  emphasize  that  an  interview  guide  is  quite  different 
from  a  questionnaire  in  that  an  interview  guide  serves  primarily  to  remind  the  interviewer 
of  the  intent  of  the  questions  being  asked.  Because  of  the  complex  nature  of  the  subject 
matter,  it  was  often  necessary  for  the  interviewer  to  explain,  in  his  own  words,  the  mtetV 
of  the  questions  to  the  interviewee.  The  structured  portion  of  the  guide  served  two 
purposes:  first,  it  allowed  for  categorizing  the  responses  o  certain  questions;  second,  it 
allowed  for  simplified  retonhug  of  the  responses  directly  on  the  guide  itself  by  entering 
a  single  letter  or  number  in  a  designated  block.  The  unstructured  portion  of  the  guide 
allowed  the  intern  ewer  to  probe  particular  subject  areas  in  greater  depth,  and  record  the 
responses  in  narrative  rather  thaw  coded  term.  The  subjective  data  produced  has  proved  to 
!.  '  very  useful  in  ideniitying  new  or  unexpected  plumometta. 

An  interview  guide  handbook  was  developed  ami  compiled  both  to  ;'id  m  the 
training  ot  interviewers,  and  as  a  working  t**ol  to  help  the  interviewer  in  actually  coeekietbig 
depth  interviews  A  complete  copy  ol  tin?  interview  guide  and  handbook  is  included  as  Jsee- 
lion  IV  in  Volume  11  ot  this  report  The  interview-  guide  handbook  contains  sections  on  how- 
lo  set  up  interview  appointments  and  I  tow  to  conduct  the  interview  its  el  I  **  also  includes 

sin.iplemental  material  needed  to  explain  or  clarify  particular  questions.  Tor  example,  it 
includes  a  list  ot  specialized  information  and  data  centers.  Hie  handbook  also  explains  the 


A 
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purpose  of  each  question  in  the  interview  guide  and  provides  necessary  back-up  material 
sometimes  needed  by  the  interviewer  to  communicate  the  intent  of  the  question  and  the 
meaning  of  the  answer  categories  to  the  interviewee. 

2.6  INTERVIEWER  SELECTION  AND  TRAINING 

It  was  deemed  essential  that  mature  and  personable  people,  having  a  scientific 
ot  technical  l>ackground,  should  be  utilized  as  interviewers;  therefore,  they  were  selected 
on  the  basis  of  the  following  qualifications: 

(1)  A  scientific  or  technical  background. 

(2)  A  high  degree  of  intelligence. 

(3)  A  high  degree  of  responsibility. 

(4)  A  pleasing  personality. 

(5)  Ability  to  quickly  establish  rapport. 

(6)  A  high  degree  of  self-sufficiency  and  the  ability 
to  direct  a  conversation. 

It  was  necessary  to  thoroughly  train  the  interviewing  team  in  the  purpose  of 
the  survey  and  the  actual  technique  to  be  employed.  In  the  initial  stages  the  training  was 
extensive.  As  the  training  became  more  highly  formalized,  the  interviewers  ware  able 
to  be  trained  in  a  two- week  period.  The  two-week  program  consisted  of  four  days  of  fonr.u 
classroom  training  (including  demonstration  interviews), and  five  days  of  practice  inter¬ 
viewing  in  the  field,  under  observation  of  the  training  specialist  and  usually  one  other  stu¬ 
dent  interviewer. 

2.7  PILOT  TESTS 

A  series  of  test  interviews  was  conducted  to  test  all  aspects  of  the  interview* 
gukle  and  interview  procedures. 

A  pre-pilot  survey  consisting  of  IS  test  interviews  was  conducted  with  technical 
personnel  at  At/XliBACH.  the  Kranklord  Arsenal,  and  the  Philadelphia  Xavy  Yard.  The 
actual  pilot  survey  consisted  of  73  interviews  with  personnel  selected  by  OOP,  all  of  whom 
were  located  in  the  Washington  area. 

The  |dlot  tests  resulted  in  improvements  to  all  aspects  of  the  survey  technique, 
and  in  »t*niiicanl  modifications  to  the  interview  guide  and  handbook. 
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An  important  result  of  the  pilot  test  was  the  conclusion  that  specific  questions 
relating  to  current  awareness  patterns  should  not  be  included  in  the  survey.  It  was  found 
that  such  questions  were  generally  incompatible  with  the  critical  incident  method  being 
utilized  to  identify  task  related  information  requirements. 

2.8  SAMPLING  PLAN 

The  sampling  approach  taken  during  Phase  1  was  to  select  a  simple,  random 
sample  of  approximately  four  percent  of  the  RDT&E  population.  The  procedure  first 
involved  sequential  numbering  ol  38,000  population  cards,  taking  3,000  random  numbers 
from  a  table  of  random  numbers,  and  selecting  the  cards  in  the  population  deck  which 
matched  the  random  numbers.  After  eliminating  duplicates,  about  2,700  cards  remained. 

To  reduce  this  number  to  the  desired  sample  size,  approximately  one-half  of  the  resulting 
2,700  cards  were  randomly  chosen.  The  sample  cards  (selected  in  the  above  manner) 
were  sorted  by  installation  for  convenience  in  scheduling. 

While  other  sampling  techniques  might  have  provided  greater  economy  or 
precision,  the  simple  random  sampling  method  was  selected  for  very  good  reasons.  The  major 
reason  was  that  sufficient  data  describing  the  population  characteristics  were  not  readily  avail¬ 
able  to  allow  for  any  meaningful  stratification  of  the  sample.  As  a  result  of  using  a  random 
sample,  a  comprehensive  picture  of  the  RDT&E  population  was  able  to  be  developed.  Further, 
the  sample  size  was  sufficiently  large  that  the  information  patterns  of  meaningful  subsets  of 
the  population  were  able  to  be  analyzed. 

2 •  9  SURVEY  MANAGEMENT 

A  number  of  significant  administrative  problems  had  to  be  solved  to  facilitate 
the  conduct  of  the  study,  including  efficient  scheduling,  developing  effective  means  for 
obtaining  the  cooperation  of  those  to  be  interviewed,  handling  missed  interviews,  and  the 
like.  These  are  discussed  in  Appendix  A. 

2 .  LI  CONDUCT  OF  INTE R VIEWS 

The  survey  consisted  of  1,375  personal  in-depth  interviews  of  the  total  BOD 
RDT&E  population.  Each  interview  lasted  approximately  two  hours.  The  interviews  were 
conducted  in  private  to  assure  confidentiality  and  to  prevent  bias. 

The  principal  parts  of  tho  interview  were; 

(1)  Information  pertaining  to  the  respondent. 

(2)  Task  definition  and  isolation. 


(3)  Identification  of  required  chunks  of  information. 

(4)  Questions  concerning  the  acquisition  and  use  of  chunks. 

In  addition  to  these  principal  parts,  other  questions  were  asked  about  the 
respondent's  use  of  certain  formal  information  services  and  questions  concerning  his 
general  ability  to  obtain  information.  These  questions  were  asked  toward  the  end  of 
the  interview  and  were  intended  to  measure  the  use  of  DOD  formal  information  services, 
as  well  as  to  determine  the  respondent*  s  satisfaction  with  the  services  available  to  him. 

2. 11  AUDIT  AND  QUALITY  CONTROL 

To  assure  consistent  high  quality  data,  a  number  of  audit  and  quality  control 
provisions  were  made.  First,  all  of  the  completed  interview  guides  were  reviewed  for  com¬ 
pleteness,  consistency,  overall  logic,  and  the  appropriateness  of  the  alphanumeric 
coding.  In  addition,  occasional  field  spot  checks  of  actual  interviews  were  made  by  the 
analyst,  tire  training  specialist,  or  the  project  engineer.  As  problems  were  encountered, 
feedback  was  provided  to  the  interviewers  with  suggestions  for  their  solution. 

The  interview  guide  itself,  and  the  computer  programs  for  file  maintenance, 
provide  a  number  of  edit  or  self-checking  features.  The  guide  was  designed  to  facilitate 
keypunching.  In  addition,  100  percent  of  the  coded  data  was  key  verified  to  ensure 
against  keypunching  errors. 

2.12  DATA  COMPILATION  SYSTEM 

A  series  of  six  computer  programs  was  developed  for  compiling,  analyzing, 
compressing,  and  presenting  the  data.  In  particular,  a  data  retrieval  system  was 
provided  which  will  allow  for  various  types  of  questions  to  he  asked  of  the  data  base. 

For  example,  one-way,  two-way,  and  three-way  tables,  which  combine  the  answers 
to  specific  combinations  of  questions,  can  be  provided  and  printed  out  in  a  useful 
format.  In  addition,  questions  can  be  asked  of  the  data  base  with  provisions  for  AND, 

OU,  and  NOT  operators.  This  provides  lor  answering  such  questions  as  "How  many  inter¬ 
viewees  answered  YES  to  question  13  (the  task  was  assigned)  and  A  (received  with  task 
itssignment)  to  question  40?" 

Thus,  it  is  possible  to  look  at  the  Interrelationships  among  the  answers  to  various 
questions  by  each  individual  interviewee  as  well  as  the  relationship  of  the  total  distribution 


of  a  given  question  to  the  total  distribution  of  another  question.  Since  the  programs 
also  provide  for  a  printout  of  the  specific  interview  guide  numbers,  which  satisfy  a 
particular  intersection,  union,  or  negation  inquiry,  it  is  also  possible  to  go  back  into 
the  completed  interview  guides  for  further  analyses. 


2. 13  ANALYSIS  OF  DATA 

The  data  was  compiled  in  three  increments,  upon  completion  of  474,  930, 
and  1,373  interviews.  The  computer  data  compilation  system  summarized,  sorted, 
and  formatted  the  data.  Various  presentations  of  he  data  were  printed  out  in  the  form 
of  simple  frequency  distributions  with  associated  percentages,  two  and  three-way  tables, 
and  the  answers  to  specified  intersection  questions  which  showed  the  inter-relationship 
between  the  answers  to  several  questions. 


The  data  resulting  from  each  survey  question  was  evaluated  in  terms  of  its 
reliability  and  freedom  from  bias.  Such  factors  as  how  easily  the  intent  of  the  question 
could  be  communicated  to  the  interviewee,  whether  the  answer  categories  proved  to  be 
mutually  exclusive  and  ail  inclusive,  and  the  subjectivity  of  the  question,  entered  into 
the  evaluation.  A  rating  scheme  was  developed  using  a  simple  index  of  1  4  to  indicate 

the  quality  of  the  data  obtained  by  each  question.  A  rating  of  l  indicated  excellent 
quality,  whereas  a  rating  of  4  indicated  data  having  very  marginal  reliability.  If  the 
data  from  a  particular  question  was  considered  to  be  of  marginal  reliability,  little 
further  analysis  was  per! armed  on  this  question. 


The  procedure  by  which  the  various  two-way  and  three-way  tables  and  inter¬ 
section  inquiries  were  selected  was  as  follows:  first,  the  questions  wore  grouped  into 
four  categories  —environment  (characteristics  of  the  person  and  the  task),  responses 
(characteristics  of  the  information  required  by  tho  task),  performance  (characteristics 
of  the  inlormation  actually  obtained  for  the  task,  i.e.,  timeliness,  depth,  source,  volume, 
etc.),  ami  narrative  questions. 


Next,  the  possibility  of  the  existence  of  a  significant  relationship  between  the 
subjects  of  questions  within  one  category  and  tho  subjects  of  questions  within  a  second 
category  was  examined.  Possible  relationships  between  the  subjects  of  two  or  more 
question  within  a  category  were  also  examined.  The  hypotheses,  underlying  the  possible 
existence  of  significant  relationships,  were  developed  jointly  by  DOD  and  AUERBACH. 
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On  the  basis  of  the  possible  relationships  hypothesized,  approximately  800 
two-way  and  three-way  tables  were  compiled  and  printed  out  by  the  computer  system. 
Where  appropriate,  they  were  subjected  to  standard  chi-squared  tests  at  the  95  percent 
confidence  level  to  determine  whether  or  not  there  is  a  statistically  significant  rela¬ 
tionship  between  the  subjects  of  two  questions.  Where  a  significant  relationship  was 
found,  an  attempt  was  made  to  interpret  and  measure  the  degree  of  relationship. 

A  more  detailed  description  of  the  analysis  procedure  is  provided  in 
Appendix  A.  Section  in  presents  some  of  the  results  of  the  analysis.  A  selection  of 
the  more  significant  two-way  and  three-way  tables  with  brief  interpretive  comments 
is  presented  in  Section  VI  of  Volume  II. 
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SECTION  III.  ANALYSIS  OF  DATA 


3.1  INTRODUCTION 

The  survey  produced  data  in  each  of  the  following  categories  of  the  Interview 

Guide: 

(1)  Characteristics  of  the  POD  RDT&E  Population; 
e.g. ,  age,  education,  job  title,  rating,  MOS  or 
job  code,  type,  kind,  and  field  of  activity. 

(2)  Characteristics  of  RDT&E  Tasks;  e.g. ,  field, 
length  of  task,  how  originated,  kind,  form,  and 
nature  of  output. 

(3)  Characteristics  of  the  Information  Chunks  Acquired 
and  Used  in  Performing  RDT&E  Tasks;  e.g. ,  class, 
media, volume,  depth,  timeliness. 

(4)  Use  of  Information  Services;  e.g. ,  TAB,  DDC, 
information  centers. 

The  RDT&F.  population  was  found  to  be  quite  heterogeneous  in  that  it  is  engaged 
in  a  wide  variety  of  tasks  of  varying  scope  throughout  a  broad  range  of  fields.  The  findings 
described  in  Section  1  thereby  relate  to  a  typical  randomly  selected  DOD  technical  man  and 
the  typical  tasks  in  which  he  was  found  to  be  engaged.  Consequently,  it  is  important  to 
examine  whether  the  findings  are  uniform  throughout  all  segments  of  the  population,  or 
whether  a  significant  variation  exists  for  a  given  type  of  person  or  task.  For  example,  do 
people  engaged  in  research  tasks  require  performance  and  characteristics  data  as  fre¬ 
quently  as  people  engaged  in  development  work? 

The  analysis  methods  described  in  Paragraphs  ‘2. 13  and  A.  5. 3  of  Appendix  A 
were  utilized  to  identify'  the  existence  or  absence  of  significant  variations  in  the  iindings. 
These  variations  are  described  in  this  Section.  First,  the  characteristics  of  the  popula¬ 
tion  (e.g. ,  field,  education,  type,  and  kind  of  activity)  are  compared  against  each  other, 
against  task  characteristics,  and  with  the  use  of  information  services.  Second,  the  char¬ 
acteristics  of  the  information  chunks  (e.g. ,  class,  media,  depth,  timeliness,  volume) 
used  in  performing  RDT&E  tasks  are  compared  to  one  another,  and  are  then  compared  to 
various  characteristics  of  the  tasks  (e.g. ,  field,  kind,  form,  and  nature  of  output)- 


The  findings  presented  in  Section  I  were  essentially  based  on  the  one-way  dis¬ 
tributions  of  the  answers  to  individual  questions.  All  of  the  one-way  distributions  for  each 
of  the  survey  questions,  both  pooled  and  unpooled,  are  presented  without  comment  in 
Section  VI  of  Volume  n.  Appendix  B  of  this  volume  contains  a  discussion  of  each  survey 
question  and  its  associated  one-way  table.  The  reader  is  encouraged  to  refer  to  Appendix 
B  for  an  explanation  of  the  intent  of  each  question,  the  statistical  results,  and  a  discus¬ 
sion  of  any  problems  which  may  have  affected  the  quality  of  the  data. 

To  make  the  comparisons  presented  in  this  Section,  it  was  necessary  to  analyze 
not  only  the  simple  one-way  distributions  of  the  answers  to  the  survey  question,  but  also 
two-way  and  three-way  distributions  which  combine  the  answers  to  two  or  three  questions. 

Approximately  600  tables  were  analyzed  to  determine  possible  relationships 
between  the  subjects  of  the  various  survey  questions.  A  small  sampling  of  these  tables 
is  included  in  this  Section  for  ease  of  presentation.  In  several  of  these  tables,  the  data 
in  particular  cells  has  been  circled  to  highlight  that  portion  of  the  table  which  is  of  primary 
interest  to  the  pattern  identified.  Frequently,  the  only  significance  of  a  particular  table 
is  that  there  is  no  outstanding  pattern  or  relationship  between  the  subjects  of  the  two 
questions  being  examined.  In  most  cases,  the  reader  is  referred  to  Section  VI  which 
contains  all  of  the  one-way  tallies,  and  a  selection  of  the  most  significant  two-way  and 
three-way  tables  including  brief  annotations  describing  tlioir  significance. 

Figure  3-1  is  an  example  of  a  two-way  table  with  an  explanation  of  how  to  read 
and  interpret  the  row  and  column  figures.  The  tablo  number  (0950),  which  appears  at  the 
upper  left  hand  corner  of  the  table,  refers  to  the  answers  to  Question  9  (Type  of  Activity) 
distributed  against  the  answers  to  Question  50  (Do  You  Use  DDC?).  laxiking  at  this 
sample  table  in  Figure  3-1,  wo  can  examine  it  in  several  ways.  First,  we  can  compare 
the  individual  cell  percentages  against  the  column  total  percentage.  For  example,  vliile 
*1?  percent  of  the  total  population  use  DDC,  only  35  percent  of  the  people  engaged  in  tech¬ 
nical  administration  use  DDC.  Comparing  the  cell  percentages  against  the  row  totals, 

we  see  that  57  percent  of  the  total  pollution  are  engaged  in  detailed  scientific  or  engin- 

* 

coring  activities,  and  tliat  58  percent  of  those  people  who  use  DDC  are  in  detailed 
scientific  or  engineering  activities. 
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TYPE  OF  ACTIVITY  VS.  USE  OF  DDC 


Question  9 

TYPE  OF 
ACTIVITY 


Detailed  Scientific 
or  Engineering _ 


Technical 

Evaluation 


Technical 
Administration . 


Question  50  Do  you  use  DDC  (ASTIA) r. 
Yes  No 

| - J - 


206 

32 


192 

26 


C  35  63 

55  10G 

9  14 


17 


3 

50 


399 

29 


100 


152 

11 


Other 


D  19  75 

6  24 

1  3 


.<0  DF 
TOTAL  47 


53 

640  729 

100  100 


2 

33 


6 

100 


100 

32 

2 

1 

100 

_ 1.375 

100 


EXPLANATION  OF  THE  CALL  OUTS 

The  UU'ce  vuiiius  ill  each  cell  arc-  as  fellows  {refer  In  fell  UA-50AI: 

<£>  373  is  the  cell  frequency;  i.e. ,  out  of  1375  responses, 

373  were  in  category  A  in  question  9  and  also  in 
category  A  in  ques'ion  30. 

<S>  47  indicates  that  *hia  cell  contains  17  percent  of  all  the 

v  entries  for  that  rjw  •  *  (373/7St»  j.  ion  at i 

3s  indicates  that  this  cell  contains  3s  percent  ot  all  the 
entries  for  that  column  (373/040  \  jou  5s). 

Similarly  for  row  totals  (refer  to  ;ow  >A). 

“sfi  is  the  sum  of  the*  cells  in  that  row  (373  ■  1 13  7s«) 

<(&  100  is  the  sum  of  the  cell  row-hared  percentages  in¬ 

cluding  round-off  error  which  appears  in  the  eelumn 
(or  row)  marked  HD  DF 

<£>  This  number  indicates  that  57  percent  of  the  population 
v  (7sh/l3T5  x  l Cm  57)  answered  question  9  ir,  eategory 
A -(Detailed  Scientific  or  Engineering) 

For  Value  of  Questions  (set*  upper  left-hand  rtiiwrl  « 

Next  to  the  question  number  t«  a  one  digit  number  m'I 
v  off  by  dashes  (e.g. ,  -2-1  This  number  represents 
the  composite  evaluation  which  has  been  placed  uihsu 
the  reliability  of  the  data  contained  in  each  table. 

A  rating  of  -  I  -  indicate*  that  the  quality  of  the  data  is 
excellent,  whereas  a  rating  ei  -  4  >  indicates  Uut  the 
ilata  has  very  marginal  reliability. 


Figure  o-l.  Kxutupie  of  A  Two-Way  Table 
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CHARACTERISTICS  OF  THE  DOD  RDT&E  POPULATION 


Questions  1-11  are  concerned  with  various  characteristics  of  RDT&E  per¬ 
sonnel,  including  MIL/GS  rating,  Highest  Degree  and  Field,  MOS  or  Job  Code,  and  Type, 
Kind  and  Field  of  Activity.  The  reader  is  referred  to  Appendix  B  for  a  discussion  of  the 
one-way  distributions  of  the  responses  to  these  questions. 

Personnel  Characteristics  vs.  Task  Characteristics 

•  Table  0112  (MIL/GS  Rating  vs.  Field  of  Task)  shows  that  there  are  no 
relationships  between  the  fields  of  the  tasks  and  the  ratings  of  DOD  RDT&E  personnel, 
with  one  exception;  there  appear  to  be  relatively  more  higher  rated  people  performing 
bisks  in  the  Aircraft  and  Flight  Equipment  field  than  in  other  fields. 

•  Table  Oil  i  (MIL/GS  Rating  vs.  Kind  of  Task)  shows  that  higher  rated 
people  appear  to  be  performing  relatively  more  tasks  in  R&D  support  than  in  other  kinds 
of  tasks.  Somewhat  surprisingly,  the  table  does  not  show  that  research  tasks  are 
normally  performed  by  people  with  higher  than  average  ratings.  Table  0414  (Highest 
Degree  vs.  Kind  of  Task)  shows  that  people  with  engineering  degrees  were  heavily  en¬ 
gaged  in  engineering  development  and  reliability  tasks,  whereas  people  with  scientific 
degrees  tended  to  be  engaged  relatively  more  often  in  research  tasks.  This  result  is 
probably  accounted  for  by  the  fact  that  research  tasks  tend  to  be  of  relatively  longer 
duration  than  other  tasks. 

«  Table  0912  (Type  ol  Activity  vs.  Field  of  Task)  shows  that  in  the  field  of 
Aircraft  and  Flight  Equipment,  relatively  more  people  were  engaged  in  technical  admin¬ 
istration  and  fewer  in  detailed  scientific  and  engineering  work,  as  compared  to  other 
fields. 


*  Table  0914  (Type  of  Activity  vs.  Kind  of  Task)  shows  that  research  tasks 
required  primarily  detailed  scientific  and  engineering  people  and  proportionately  less 
teclmieal  evaluation  or  administrative  type  people  than  did  other  kinds  of  tasks. 

* 

•  Table  1012  (Kind  of  Activity  vs.  Field  of  Task)  shows  that  research  people 
were  doing  relatively  fewer  tasks  in  the  fields  of  Aircraft  and  1  light  Equipment  and  more 
in  the  fields  of  Medicine  and  Physics,  Fluid  Mechanics,  and  Nuclear  Physics.  Conversely, 
engineering  people  were  found  to  l>e  performing  relatively  fewer  tasks  in  the  fields  of 
Aircraft  and  flight  Equipment,  Guided  Missiles,  and  Medicine  and  mure  in  the  field  of 
Research  and  Research  Equipment. 


3“ 4 


•  Table  1014  (Kind  of  Activity  vs.  Kind  of  Task)  shows  a  very  heavy  diag¬ 
onal  as  would  be  expected,  since  the  definitions  of  the  categories  are  the  same  for 
Question  10  (Kind  of  Activity)  as  they  are  for  Question  14  (Kind  of  Task).  This  pattern 
implies  that  people  normally  engaged  in  a  particular  kind  of  activity  were  found  to  be 
engaged  in  the  same  kind  of  task. 

•  Table  1112  (Field  of  Activity  vs.  Field  of  Task)  also  shows  a  heavy 
diagonal  line  which  implies  that  the  respondents'  fields  of  endeavor  tend  to  be  the  same 
as  the  fields  of  the  tasks  in  which  they  are  engaged. 

Personnel  Characteristics  vs.  Information  Found  After  Task  Completion 

•  Table  0148  (MIL/GS  Rating  vs.  Post  Task  Information)  shows  that  there 
is  no  relationship  between  a  person's  rating  and  whether  or  not  he  has  found  some 
pertinent  but  previously  unknown  information  after  the  completion  of  the  same  task. 

•  Table  1048  (Kind  of  Activity  vs.  Post  Task  Information),  Figure  3-2,  and 
Table  1148 (Field  of  Activity  vs.  Post  Task  Information),  however,  show  that  research 
people  tend  to  find  relevant  information  after  task  completion  more  often  than  other 
types  of  personnel.  This  is  true  also  for  personnel  engaged  in  mathematical  work. 

People  engaged  in  the  field  of  Ordnance  rarely  found  relevant  information  after  task 
completion. 

MIL/GS  Rating  vs.  Typo  of  Activity 

•  Table  0109  (MIL/GS  Rating  vs.  Type  of  Activity)  shows  that  people  engaged 
in  detailed  scientific  and  engineering  effort  tend  to  have  lower  Job  ratings  than  people  in 
technical  evaluation  ami  in  technical  administration. 

Personnel  Characteristics  vs.  Use  ol‘  Information  Centers 

•  Table  U49  (Field  of  Activity  vs.  Use  of  TAB)  shows  tint  people  engaged  in 
the  fields  of  Aircraft  and  Flight  Equipment,  and  Materials  and  Metallurgy  used  TAB 
relatively  more  than  people  in  other  fields.  On  the  other  hand ,  Table  1150  (Field  of 
Activity  vs.  Use  of  DDC)  shows  that  people  engaged  in  tile  fields  of  Fuels  and  Propulsion 
Systems,  and  in  the  fields  of  Physics,  Fluid  Mechanics,  Nuclear  Physics,  use  DDC  rela¬ 
tively  more  Utan  people  engaged  in  other  fields.  People  working  in  the  field  of  Medicine 
appear  to  use  DDC  relatively  less  than  individuals  in  other  fields. 


fc.q.  •? 


3-5 


f 


1048  -  2  -  KIND  OF  ACTIVITY  vs.  POST 

TASK  INFORMATION 

This  comparison  shows  that,  proportionally,  research 
people  seem  to  find  more  information  after  the  com¬ 
pletion  of  the  task  than  do  people  in  other  activities. 


Information  No  Information 
Found  After  Found  After 
Task  Was  Task  Was 
Completed  Completed 


Research 


Exploi'atory,  Advanced, 
Engineering,  Operational 
Development 


10-48  A 

\ 

A 


© 


B  (14 


Reliability  -  Quality  Control  C 


R&D  Support 


Other,  Blank 


0 


© 

® 


29 

17 


8 

5 

HO  OF  -  1 

TOTAL  (l3) 

W173 

100 


B 


81 


36 

21 


171 

15 


82 


97 

56 


553 

48 


94 


44 


85 


271 

24 


84 


109 


83 

1148 

100 


BLNK  RD  OF  TOTAL 


21 

39 


18 

33 


12 

22 


54 

100 


100 


210 

15 


100 


671 

49 


100 


47 


100 


318 

23 


100 

129 

9 

l 

100 

1375 

100 


Figure  3-2.  Kind  of  Activity  Vs.  Post  Task  Information 
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•  Table  0150  (MIL/GS  Rating  vs.  Use  of  DDC)  shown  in  Figure  3-3,  presents 
a  slight  pattern  which  suggests  that  a  person’s  use  of  DDC  increases  with  his  rating. 
Technical  evaluation  people  were  found  to  use  DDC  relatively  more  than  detailed 
scientific  and  engineering  people  or  administrative  people  (see  Figure  3-1). 

•  Table  0152  (MIL/GS  Rating  vs.  Use  of  Information  Centers)  shows  a  slight 
pattern  which  indicates  that  as  a  person's  rating  increases  so  does  his  use  of  specialized 
information  centers.  This  pattern  is  somewhat  more  pronounced  than  the  corresponding 
finding  in  the  previous  paragraph  derived  from  Table  0150. 

•  People  classified  as  Technical  Evaluation  were  found  to  use  information 
centers  relatively  more  than  people  classified  as  Detailed  Scientific  and  Engineering  or 
classified  as  Technical  Administration.  This  comes  from  a  comparison  of  the  circled 
figures  on  Table  0952  (Type  of  Activity  vs.  Use  of  Information  Centers)  shown  in  Figure 
3-4.  Table  1152  (Field  of  Activity  vs.  Use  of  Information  Centers)  show's  no  outstanding 
patterns,  which  implies  that  a  person's  field  of  activity  is  independent  of  his  use  of  in¬ 
formation  centers. 

Personnel  Characteristics  vs.  Information  Problem 

An  elimination  of  Tables  0156,  0756  (Figure  3-5)  and  1156  (MIL/GS  Rating, 
MOS  or  .lob  Coc.e,  Field  of  Activity  vs-  Presence  of  Information  Problem)  shows  that 
as  the  three  personnel  characteristics  (rating,  MOS,  and  field)  vary,  no  significant 
relationships  are  found  between  personnel  characteristics  and  the  presence  or  absence 
of  an  information  problem.  Table  0436  shown  in  Figure  3-6,  however,  displays  a  pattern 
which  implies  that  people  with  post-graduate  degrees  vend  to  express  having  a  serious 
problem  in  finding  or  obtaining  information  relatively  more  than  those  with  a  bachelor's 
degree  or  those  with  no  degree. 

3 . 3  CLASS  OF  INFORMATION  CHUNK 

Question  25  (Class  of  Information)  demonstrated  the  widespread  use  of  per- 

% 

formance  and  characteristics  and  specifications  data  in  ail  varieties  of  RDT&E  tasks. 

It  was  found  licit  of  the  total  number  of  chunks  of  information  required  in  the  sampled 
tasks,  42  percent  were  classified  as  performance  and  characteristics  and  specifications 


0150  -  2  -  MIL/GS  RATING  vs.  USE  OF  DDC 

Using  the  chi  squared  test  with  the  null  hypothesis  that 
a  person’s  rating  is  independent  of  his  use  of  DDC,  it 
was  found  that  at  the  five  percent  confidence  level  the 
hypothesis  should  be  rejected.  The  table  shows  that 
higher  rated  people  tend  to  use  DDC  and  lower  rated 
people  do  not. 


Figure  3-3,  MIL/GS  Rating  Vs.  Use  of  DDC 
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TV  PE  OF  ACTIVITY  vs.  USE  OF  INFORMATION  CENTERS 


This  table  shows  a  slight  tendency  for  technical  evaluation  people  to  use  information 
centers  relative  to  the  rest  of  the  population. 


Uses  Does  Not  Use 

Information  Information 
Centers  Centers 


9-52  A 


BLNK  RD  DF  TOTAL 


Detailed  Scientific 
or  Engineering 


Technical 

Evaluation 


Technical 

Administration 


A  (53J  46 

^^414  359 

55  59 

B  ( bZ)  38 

^246  151 


47 

81  74 

11  12 


2  -  l 

13 
62 

1  -  1 

2 

10 


786 

57 


399 

29 


158 

11 


Other 


0  28  63 

9  20 

1  3 


RD  OF  l 

TOTAL  (55)  44 

'  750  604 

100  100 


100 

1375 

100 


Figure  3-4.  Type  of  Activity  Vs.  Use  of  Information  Centers 
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MOS  OR  JOB  CODE  vs.  PRESENCE  OF  INFORMATION  PROBLEM 


This  table  shows  no  outstanding  features  other  than  that  people  engaged  in  the  mathematics-statistics 
series  seem  to  have  relatively  less  of  an  information  problem" than  those  people  in  other  job  codes. 


Has  Has  No 

Information  Information 
Problem  Problem 


Biology,  Medical 
Officer 


7-56  A 
A  33 

16 


B  SINK  R0  OF  TOTAL 


58 


28 


General  Engineering; 
Civil,  Electronic, 
Aerospace,  Marine,  ” 
Industrial,  and 
Mechanical  Engineering 

General  Physical 
Sciences;  Physics, 

Che  mist  ry ,  Metallurgy 
and  Meteorology 


8  28 


67 


192 

52 


466 

51 


C  29 


65 


87 

24 


197 

21 


Mathematics 

l  ’  nknown  Geog  ra phy , 
Psychology,  Library 
and  Archives,  R&-0 
Coordination,  and 
Navigator  and 
Photographer 


(10) 


87 


71 


8 


6  -  1 
39 
46 


17 

20 

-  1 


100 


48 


100 


697 

51 


100 


301 

22 


100 


82 


64 


67 

18 


«.  F  27 

RO  OF 

TOTAL  (27)  67 


159 

17 


21 

25 


100 


370 

100 


921 

100 


100 


24  7 
18 


100 

HI* 

IOC 


Figure  3-5.  MOS  or  Code  Vs.  Presence  ol  information!  Problem 
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0456  -  2  -  HIGHEST  DEGREE  AND  FIELD  vs. 

PRESENCE  OF  INFORMATION 
PROBLEM 

Proportionally  more  people  with  higher  degrees  con¬ 
sider  that  they  have  an  information  problem. 


Has  Has  No 

Information  Information 
Problem  Problem 

4-56  A  B  BLNK  RD  DF 


TOTAL 


No  Degree  A  23  70  7 

32  96  9 

9  10  11 

Bachelor  in  Aero,  Chemical, 

Civil,  Electrical,  Industrial,  B  26  68  7  -  1 

Mech,  Metallurgy,  General,  135  353  35 

Other,  Engineering  37  39  42 


Bachelor  in  Agriculture, Biology.  C  26 

Chemistry,  Math,  Physics, 

Psychology- 


Master  in  (all  those  in  B) 


Master  in  (all  those  in  C) 


94  259  15 

25  28  18 


D  (36)  58  7  -  1 

33  53  6 

9  6  7 


E  (32)  61  7 

43  81  9 

12  9  U 


Ph.D  in  (ail  those  in  B) 


Ph.D  in  (all  those  in  C) 


F  (50)  30  13  -  1 

4  3  1 

1  l 

C  (25)  67  9  -  1 

23  62  8 

6  7  10 


All  Medicine 


K  (33)  60  7 

^  /  5  9  1 


R0  DF  -  1 

TOTAL  (27)  67  6 

370  921  84 

100  100  '  100 


100 

1375 

100 


Figure  3-6.  Highest  Degree  und  Field  Vs^  Presence  oi  Information  Problem 


Class  of  Information  vs.  Task  Characteristics 


Figure  3-7  compares  the  answers  to  Question  25  (Class  of  Information)  to 
the  answers  to  Questions  12  (Field  of  Task),  14  (Kind  of  Task),  17  (Length  of  Task), 
and  94  (Task  Output) . 

•  Within  Figure  3-7,  the  comparison  of  Field  of  Task  vs.  Class  of  Infor¬ 
mation  shows  that  there  is  no  significant  difference  in  the  degree  of  use  of  one  class  of 
information  over  another  as  the  field  of  the  task  changes. 

•  The  comparison  between  Kind  of  Task  and  Class  of  Information  shows 
that  Research  tasks  tend  to  require  relatively  more  concept  information  and  less  per¬ 
formance  and  characteristics  information  than  do  the  other  kinds  of  tasks. 

•  Engineering  tasks  were  found  to  require  more  Performance  and  Char¬ 
acteristics  and  Specifications  data  than  other  kinds  of  tasks. 

•  A  comparison  between  the  Man-Days  of  the  Task  and  Class  of  Information 
shows  that  as  the  length  of  a  task  increases,  there  appears  to  be  a  greater  use  of  con¬ 
cept  information. 

•  A  comparison  between  the  Task  Output  and  the  Class  of  Information 
confirms  that  there  is  a  fairly  strong  correspondence  between  the  nature  of  the  output 
of  the  task  and  the  class  of  information  used  within  the  task. 

In  summary,  it  can  be  said  that  with  an  nor  exceptions,  as  the  task  character¬ 
istics  change,  there  does  not  appear  to  be  a  significant  change  in  the  classes  of  informa¬ 
tion  used  to  conclude  the  tasks.  It  is  further  observed  that  the  high  usage  of  performance 
and  characteristic  and  specifications  data,  and  "how-to-do-it"  information  (design 
techniques,  etc.)  does  not  appear  to  vary  significantly  with  differences  in  RDT&E  task 
characteristics. 

Class  of  Information  vs.  Media 

The  media  used  to  convey  information  are  described  in  Question  28  which  is 
discussed  in  Appendix  H.  The  most  popular  means  of  obtaining  information  was  by  way 
of  oral  media,  a  distant  second  being  by  way  of  proposals  and  reports  When  Question  25 
(Class  of  Information)  was  compared  to  Question  28  (Media),  the  following  patterns 
were  found  to  exist  (see  Table  2528  in  Figure  3-8). 
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(1)  Concept  information  was  conveyed  principally  by  way  of  texts,  journals, 
and  from  previous  knowledge;  relatively  little  use  was  made  of  brochures,  catalogs, 
standards,  drawings,  parts  lists,  etc.  (category  A  of  Question  28). 

(2)  Cost  and  funding  information  seemed  to  be  conveyed  relatively  more 
often  by  way  of  the  oral  media  rather  than  by  any  other  media. 

(3)  "How-to-do-it"  information  (design  techniques,  etc.)  did  not  tend  to  be 
communicated  by  any  one  particular  medium. 

(4)  Math  aids  and  formulae  seemed  to  be  obtained  principally  from  texts. 

(5)  Performance  and  characteristics,  and  specification  information  was 
obtained  to  a  great  extent  from  brochures,  catalogs,  standards,  etc.  (category  A  of 
Question  28),  whereas  very  little  performance  and  characteristics,  and  specifications 
information  was  obtained  from  texts. 

Class  vs.  Volume  of  Information 

Table  2532  (Class  of  Chunk  vs.  Acttial  Exposure  to  Information)  shows  the 
exposure  to  an  assumed  quantity  (volume)  of  information  obtained  in  the  sampled  tasks 
for  each  class  of  information.  By  examining  this  table,  it  is  found  that  for  math  aids 
and  formulae,  one  item  containing  the  information  is  usually  sufficient  in  most  cases. 
Generally,  however,  this  table  shows  no  outstanding  patterns,  implying  that  the  exposure 
to  information  does  not  vary  significantly  as  the  classes  of  information  change.  A 
comparison  between  Tables  2532  and  2533  (Class  of  Chunk  vs.  Desired  Exposure  to 
Information)  shows  the  difference  between  the  exposure  wanted  and  the  exposure  actually 
obtained.  It  is  interesting  to  note  that  there  is  relatively  no  difference  between  these 
two  tables,  possible  implying  that  the  user  either  gets  what  he  wants  or  accepts  what¬ 
ever  he  gets  (i.e, ,  he  has  learned  to  live  with  the  system). 

Class  vs.  Depth 

Table  2538  (Class  of  Chunk  vs.  Actual  Depth  of  Information)  shows  the  rela¬ 
tionship  between  the  depth  of  the  information  actually  obtained  and  the  class  of  information. 
By  examining  this  it  is  seen  that  math  aids  and  formulae,  and  raw  data  are  com¬ 
monly'  received  1  '*  the  form  of  a  specific  answer  (great  depth)  whereas  technical  status 
informs  jion  is  mo»*e  commonly  received  in  a  once-over  lightly  treatment  (limited  depth). 
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Note: 


All  figures  are  percentages. 
Information  that  was  recorded  during 
interview  as  "Other"  or  "Blank"  is 
not  included  in  these  tables;  therefore, 
the  columns  may  not  add  up  to  100%. 
All  data  is  pooled  where  appropriate. 
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Figure  3-7.  Relationships  Between  Classes  of 
Information  and  Characteristics  of  Tasks 


3-14 


2528 
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CLASS  OF  CHUNK  vs.  MEDIA 


This  table  shows  the  following  highlights:  concept  inf  or 
engineering  type  media.  Cost  and  funding  information  i 
in  texts,  manuals,  and  handbooks.  Performance  and  sp 
(engineering  media),  the  oral  mode,  and  in  proposals  at 
obtained  by  way  of  proposals  and  reports,  the  oral  mod; 


Brochures,  Catalogs, 

Standards  and  Codes,  Corr 

Drawings,  Schematics,  Live  Demonstration,  Mem 

Parts  Lists,  System  Physical  Measurement  Pers 
Specs  (QMR,  TDP...)  or  Experiment  Filet 


Oral, 

Meetings  and 
Symposia 


Directives , 
Handbooks , 
Manuals 


25-28 


Concepts 


Cost  and  Funding 

Design  Techniques, 
Experimental  Processes, 
Production  Processes, 
Utilization,  Test 
Processes 

Math  Aids  and 
Formulae 


Performance  an  1 
Cha  raeteristics. 
Specifications 

Haw  Data 


C  II  31 

162  481 

19  21 

E  4  17 

14  63 

2  3 


29 


953 

42 


22 


73 


C 

7 


54 

21 


32 


145 

25 


16 


135 

52 


31 
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62 

11 
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ancTspJ  concept  information  is  found  proportionally  more  in  texts  and  journals  and  less  in 
>sals  anf?  information  is  obtained  by  the  oral  media.  Math  and  formula  information  is  found 
al  modi..Jrmance  and  specification  information,  to  a  great  extent,  comes  from  a  category 

in  proposals  and  reports,  and  is  not  found  in  texts.  Status  information  is  principally 
the  oral  mode,  and  journals. 


es,  j 


Correspondence, 

r ion,  Memos,  TWX  Preprint, 

urement  Personal  Notes,  Reprint, 

Files  and  Logs  Journal 


Previous 
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ks,  i 
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31 
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Figure  3-S,  Class;  of  Chunk  Vs.  Media 
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Any  significance  attached  to  the  patterns  of  information  in  this  table,  however,  must 
be  qualified  by  the  low  reliability  value  (3)  of  the  data  produced  by  Question  38. 

Class  vs.  Field  of  Task  vs.  Field  of  Chunk 

The  three-way  Table  251226  (Class  of  Chunk  vs.  Field  of  Task  vs.  Field  of 
Chunk)  shows  a  high  degree  of  correspondence  between  the  field  of  the  task  and  the 
field  of  the  classes  of  information  used  within  the  same  tasks.  This  is  evident  by  the 
pronounced  diagonal  throughout  each  of  the  sub-tables,  which  shows  the  interdisci¬ 
plinary  aspects  of  information.  The  degree  of  this  relationship,  however,  was  found 
to  vary  somewhat  with  different  classes  of  information.  For  example,  tasks  in  a  given 
field  tend  to  use  cost  and  funding  information  almost  exclusively  in  the  same  field  as 
the  tasks.  However,  tasks  using  "how-to-do-it"  information  (design  techniques,  ex¬ 
perimental  production  and  test  processes,  utilization)  were  found  to  use  a  fair  amount 
of  "how-to-do-it"  information  in  fields  other  than  the  field  of  the  task.  The  same 
pattern  was  also  observed  for  tasks  using  a  high  percentage  of  performance  and  char¬ 
acteristics,  and  specifications  information;  i.  e. ,  a  considerable  amount  of  performance 
and  characteristics  and  specifications  information  was  used  in  fields  other  than  the 
field  of  the  task. 

3.4  FIRST  SOURCE  OF  TASK -REQUIRED  INFORMATION 

The  results  of  Question  42  (first  source)  demonstrated  the  significance  of 
the  local  environment  (or  informal  information  system)  and  the  infrequent  reliance  on 
libraries  and  information  centers  as  a  first  source  of  information.  The  one-way  table 
for  Question  42  is  rcproduccn  oeiew.  .Ya  explanation  of  the  frequency  column  and  the 
blank  category  for  this  anti  other  one-way  tables  is  found  in  the  introduction  to  Appendix  11. 


Question  42 

What  was  the  first  organization  or  person  you  went  to 
in  ottler  to  obtain  this  information? 


A. 

Received  with  Task  Assignment 

Frequency 

496 

Percent 

li 

11. 

Supervisor  (if  Given  by  Supervisor  after 
Assignment) ,  Assignment  to  Subordinate 

26S 

6 

E. 

Consultants,  Colleague 

102? 

22 

F. 

Librarian  or  Technical  Researcher,  Library 
(Search  by  Self) 

242 

5 
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H. 

Department  Bookcase  or  Files 

Frequency 

608 

Percent 

13 

I. 

Own  Collection 

820 

17 

J. 

Information  or  Data  Centers 

19 

0 

K. 

Manufacturer  or  Supplier 

215 

5 

L. 

Blank 

991 

21 

Total 

4687 

First  Source  vs.  Task  Characteristics 

The  first  source  to  which  the  respondent  went  to  acquire  task-required  infor¬ 
mation  was  examined  in  relation  to  the  characteristics  of  the  tasks. 

•  Table  1242  (Field  of  Task  vs.  First  Source)  shows  that  there  is  no  sig¬ 
nificant  pattern  between  the  field  of  a  task  and  the  use  of  any  particular  first  source, 
with  the  one  exception  that  for  Aircraft  and  Flight  Equipment  tasks,  manufacturers 
were  used  as  a  first  source  more  often  than  in  other  fields. 

•  Table  1442  (Kind  of  Task  vs.  First  Source)  in  Figure  3-9  further  shows 
that  as  tasks  are  classified  in  another  manner,  there  is  still  no  outstanding  relationship 
to  a  particular  first  source,  with  one  exception;  libraries  were  used  as  a  first  source 
of  information  for  research  tasks  relatively  more  often  than  for  any  oilier  category  of 
tasks. 


•  Table  1742  (Man-Days  of  Task  vs.  First  Source)  shown  in  Figure  3-10 
compares  the  level  of  effort  in  a  task  against  the  use  of  the  various  first  sources,  which 
again  tends  to  confirm  that  the  use  of  a  particular  first  source  is  not  dependent  on  tusk 
characteristics. 

First  Source  vs.  Class 

When  classes  of  information  (Question  25)  are  compared  to  the  use  of  the 
various  first  sources  (Table  2542) ,  it  is  found  that  there  are  no  outstanding  iiatterns 
between  the  use  of  any  particular  first  source  and  Uie  various  classes  of  information. 

Two  minor  features,  however,  do  stand  out;  first,  libraries  seem  to  be  used  relatively 
more  as  a  first  source  of  concept  and  technical  status  information:  and  second  a  person's 
own  collection  seems  to  be  used  relatively  more  as  the  first  source  for  madi  aids  and 
formulae  tlian  other  first  sources. 
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Figure  3-9.  Kind  of  Task  Vs.  First  Source 
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Figure  3-10.  Man-Days  of  Task  Vs.  First  Source 


Performance  of  the  First  Source 


Question  45  determined  what  was  obtained  from  the  first  source,  e.  g. ,  all 
of  the  information,  part  of  the  information,  a  reference  to  further  information,  or 
nothing.  In  39  percent  of  the  cases,  the  first  source  provided  all  of  the  information 
required. 


•  Table  4245  (First  Sour  ce  vs.  Information  Obtained  from  First  Source) 
shows  that  where  the  First  Source  was  a  colleague,  a  supervisor,  or  a  subordinate,  the 
information  obtained  was  frequently  a  reference  to  further  information,  rather  than  all 
or  part  of  the  information  required. 

3.5  VOLUME  OF  INFORMATION  (Exposure) 

In  the  great  majority  of  cases,  the  user  does  not  want  to  be  exposed  to  all  of 
the  relevant  information  which  is  available.  The  user  is  generally  satisfied  either  with 
a  single  item  of  material  or  a  sampling  of  the  available  material.  The  one-way  table 
for  Question  33  is  presented  below. 

Question  33 


Concerning  the  total  amount  of  material  available,  which 
potentially  contains  information  on  this  chunk,  how  much 
of  it  did  you  want  to  be  exposed  to? 


A 

One  Item  of  Mater  al 

Frequency 

1611 

Percent 

34 

B 

A  Sampling  of  the  Available  Materials 

1240 

25 

c. 

All  Available  Material 

1050 

22 

D. 

Blank 

784 

17 

Total 

4867 

The  17  percent  under  "Blank**  is  composed  almost  entirely  of  those  information 
chunks  which  the  respondents  recalled  from  their  own  previous  experience. 

M 

Volume  vs.  Kind  of  Task 

Table  1433  (u)  (Kind  of  Task  vs.  Exposure  Lo  information  Wa  n  ted  -  Unpool  ed) 
presents  no  outstanding  pattern  between  the  kind  of  tasks  and  the  volume  of  information 
wanted  for  the  chunks  of  information  used  in  concluding  these  tasks. 


3*20 


Volume  vs .  Search  Aids 


It  might  be  assumed  thct  people  who  want  to  be  exposed  to  a  large  quantity  of 
information  would  at  the  same  time  want  to  use  some  type  of  search  aid  to  help  them  in 
selecting  the  most  appropriate  items  to  examine  in  detail.  Table  3334  (Desired  Exposure 
to  Information  vs.  Use  of  Search  Aids)  did  not  present  any  significant  patterns.  The 
relatively  large  numbers  found  in  row  C  of  this  table  are  due  to  the  fact  that  Question  34 
(Search  Aids)  was  asked  only  of  loose  who  indicated  a  need  for  a  large  volume  of  infor¬ 
mation. 


Volume  vs.  Depth 

Table  3339  (Desired  Exposure  to  Information  vs.  Desired  Depth  of  Information) 
shows  that  when  people  want  to  be  exposed  to  one  item  of  the  available  information,  they 
generally  appear  to  be  searching  for  a  specific  answer.  There  are,  however,  no  other 
significant  patterns  between  the  information  exposure  wanted  and  the  desired  depth  of 
information. 


3.6  SEARCH  AIDS 

Question  34  was  designed  to  determine  whether  search  aids  were  us  'd  or  de¬ 
sired  by  those  who  needed  a  large  quantity  of  information.  The  question  was  not  asked 
of  those  who  did  not  indicate  a  need  tor  a  large  volume  of  information,  since  it  was 
presumed  that  in  this  situation  search  aids  would  not  have  been  required.  This  accounts 
for  the  large  number  of  blanks  (no  answer)  in  the  one-way  distribution  shown  below.  Of 
the  19  percent  of  the  eluniks  of  information  for  which  affirmative  answers  were  obtained 
(e.  g. ,  A,  B,  or  F.  )*  search  aids  would  have  boon  useful  or  were  already  being  used  in  68 
percent  of  these  cases* 


Question  34 


Since  you  wanted  to  see  a  lot  of  material,  would  you  have  found 
title  listings  or  abstracts  more  useful  to  road  first  in  order  to 
help  you  select  the  chuuK  of  material  to  read  in  detail  ? 


Frequency 

Vercent 

A. 

No.  Wanted  to  review  ail  the  material 
Explain 

263 

6 

B 

Title  Listing,  Abstracts,  or  both 

466 

10 

F. 

Already  Used  Either  or  both 

153 

3 

X 

Blank 

3S03 

81 

Total 

4687 
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Search  Aids  vs.  Field  of  Task 


From  Table  1234  (Field  of  Task  vs  Search  Aids) ,  it  was  found  that  search 
aids  were  not  generally  considered  to  be  useful  by  those  engaged  ir.  tasks  in  the  field 
of  Communi cations  and  Electronics.  Conversely,  search  aids  were  more  heavily  used 
in  tasks  in  the  fields  of  Medicine,  Physics,  Fluid  Mechanics,and  Nuclear  Physics. 

3.7  DEPTH  OF  INFORMATION 

Questions  38  and  39  are  companion  questions  dealing  with  the  depth  of  infor¬ 
mation  received  and  wanted,  respectively.  Depth  was  defined  as  either  a  "Once-Over- 
Ughtly"  treatment  of  the  subject  or  as  a  "Detailed  Analysis. "  A  third  category  "Specific 
Answer"  was  treated  as  a  subcategory  of  "Detailed  Analysis"  (considerable  depth).  The 
significant  finding  was  that  the  RDT&E  population  wanted  either  a  Detailed  Analysis  or 
Specific  Answer  in  the  great  majority  of  cases. 

Question  38 

Depth  of  Information  Received.  Did  you  get  a: 


Frequency 

Percent 

A 

Once  over  lightly  of  the  subject 

859 

18 

B. 

Detailed  Analysis 

1994 

43 

C 

Specilie  Answer 

1002 

21 

D 

Nothing 

42 

1 

Kink 

Blank 

790 

17 

Total 

•*08? 

Question  39 

Depth  of  Information  Wanted,  Would  >ou  want  a: 

Frequency 

Percent 

A. 

Once  over  lightly  of  Use  subject 

700 

n 

£ 

Detailed  Analysis 

2152 

c. 

Specific  Answer 

1043 

'A'-’ 

id*** 

Blank 

793 

IT 

1W 

40S7 

Varying  task  characteristics  were  compared  to  the  depth  of  information  de¬ 
sired  for  the  required  chunks  of  information.  Tasks  in  the  fields  of  Aircraft  and  Flight 
Equipment  appear  to  require  only  a  once-over -lightly  treatment  relatively  more  often 
than  tasks  in  other  fields.  The  respondents  engaged  in  tasks  in  the  field  of  Medicine 
were  relatively  less  often  satisfied  with  a  once-over-lightly  treatment. 

•  Tables  1439  (u)  and  1739  (u)  (Kind  of  Task  vs.  Desired  Depth  of  Informa¬ 
tion)  show  no  significant  patterns.  This  implies  that  the  depth  of  information  wanted 
bears  little  relationship  to  task  characteristics. 

•  From  an  examination  of  Table  3942  (u)  (Desired  Depth  of  Information 

vs-  First  Source)  there  further  appears  to  be  no  relation  between  the  use  of  any  particular 
first  source  of  information  and  the  depth  of  the  information  wanted. 

3.8  RETRIEVAL  TIME 

Questions  35  and  36  are  concerned  with  how  quickly  the  information  was  re¬ 
ceived  and  how  quickly  it  was  required.  It  ms  found  that  in  only  18  percent  of  the  cases 
was  the  required  information  needed  in  less  than  one  day  or  received  with  the  task  assign¬ 
ment,  while  41  percent  of  the  information. was  actually  obtained  within  one  day(  or  received 
with  the  task  assignment). 


A. 

B. 

C. 

D. 
G. 
X. 


Question  35 


From  the  time  you  requested  this  chunk  uf  started  to 
search  for  it,  what  was  the  actual  time  it  took  to  got  it? 


No,  or  Not  Applicable 
Under  1  Day 
Under  1  Week 

Under  1  Month;  Under  3  Months;  Over  3  Months 

Received  with  task  assignment 

Blank 

Total 


Frequency  Percent 


351 

7 

lb7<> 

36 

735 

16 

895 

19 

802 

») 

)? 

4687 
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Question  36 


From  the  time  you  requested  this  chunk  or  started  to 
search  for  it,  was  there  a  maximum  time  you  could  have 
allowed  to  get  it? 


A. 

No,  or  Not  Applicable 

Frequency 

738 

Percent 

16 

B. 

tfnder  1  Day 

617 

13 

c. 

Under  1  Week 

998 

21 

D. 

Under  1  Month;  Under  3  Months;  Over  3  Months 

1315 

28 

G. 

Received  with  task  assignment 

212 

5 

X. 

Blank 

807 

17 

Total 

4687 

Ketrievai  Time  vs.  Kind  of  Task 

•  Table  1436  (u)  (Kind  of  Task  vs.  Desired  Retrieval  Time)  shows  that 
there  are  no  significant  patterns  between  the  time  information  is  wanted  and  the  cate¬ 
gories  of  tasks  found  in  Question  14  with  the  one  exception,  tint  Research  tasks  rarely 
require  information  in  loss  than  one  day. 

•  Table  1733  (Man-Days  of  Task  vs.  Actual  Retrieval  Time)  also  show's  no 
significant  patterns  between  the  length  of  die  task  and  the  actual  time  to  obtain  information. 


Retrieval  Time  vs  Volume 

*  Table  3336 (u)  (Desired  Exposure  to  Information  vs.  Desired  Retrieval  Time) 
shows  in  Figure  3-  11  that  there  are  no  significant  patterns  between  the  time  information  is 
wanted  and  die  volume  information  or  degree  of  exposure  wanted,  with  two  minor  ex¬ 
ceptions:  when  information  is  wanted  in  under  one  day,  there  seems  to  be  a  tendency  to 
examine  only  one  specific  item  of  the  available  material  (see  How  A,  Column  B,  of 
Figure  3-11)  ,  when  up  to  three  months  can  be  allowed,  there  appears  to  be  a  relatively 
greater  desire  to  in?  exposed  to  all  the  information  (see  Row  C,  Column  F  of  Figure  3-11). 


Retrieval  Time  v s .  Jtirs*^ Source 

•  Table  3342  (Actual  Retrieval  Time  vs.  First  Source)  shows  several 
jwtterns  which  exist  between  the  use  of  the  various  first  sources  ami  the  length  c<!  time 
required  to  retrieve  information.  When  the  information  is  retrieved  in  under  one  week. 
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colleagues,  manufacturers,  and  suppliers  are  used  relatively  more  as  a  first  source  of 
information.  When  it  takes  more  than  one  week  to  obtain  information,  the  library,  man¬ 
ufacturer  and  supplier,  and  supervisor  and  subordinate  are  more  frequently  used  as  the 
first  source. 
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APPENDIX  A.  STUDY  EXECUTION 


This  Appendix  contains  a  detailed  description  of  how  the  study  was  executed 
and  is  intended  to  supplement  Section  II,  which  presented  a  brief  description  of  the 
Conduct  of  the  Study.  This  Appendix  also  discusses  the  problems  of  the  study,  the 
methodology  that  was'  used  to  solve  these  problems,  and  the  reasons  for  the  decisions 
and  assumptions  that  were  made. 

The  first  part  of  the  Appendix  contains  a  discussion  of  the  overall  study  ap¬ 
proach.  This  is  followed  by  a  discussion  of  the  four  major  parts  of  the  study,  which  are: 

(1)  Survey  Techniques. 

(2)  Sampling  Plan. 

(2)  Survey  Execution. 

(4)  Compilation  and  Analysis. 

A.  1  STUDY  APPROACH 

Before  considering  the  details  of  each  phase,  it  is  useful  to  consider  some 
of  the  macroscopic  problems  of  the  task  to  obtain  a  proper  perspective  of  the  study 
since  the  magnitude  and  scope  of  tins  study  exceeds  that  of  any  other  user  studies  de- 
scu'ibed  in  the  literature.  Other  studies  have  generally  been  of  more  limited  scope  and 
have  been  confined  to  a  specific  discipline,  e.g. ,  a  survey  of  200  chemists.  Also,  most 
other  studies  have  been  concerned  with  probing  the  media  used  by  the  respondents,  with 
the  result  that  the  nature  of  the  information  itself  and  its  application  have  received  less 
attention  than  the  means  by  which  it  was  conveyed.  An  annotated  bibliography  containing 
a  description  of  n  any  of  these  studies  is  in  Appendix  C  of  tins  volume. 

A.  1.1  Su?Tey  Design 

The  available  population,  exceeding  -Ui,  oik)  KDT&K  personnel,  dispersed  over 
a  large  geographic  area,  posed  a  major  problem.  A  sample  size  exceeding  1,200  inter¬ 
views  was  required  to  gather  information  that  was  representative  of  the  total  population. 
It  was  therefore  necessary  to  devise  a  system  for  gathering,  processing,  and  analyzing 
the  results  of  the  interviews  as  efficiently  as  possible  to  work  within  a  realistic  study 
schedule.  For  tins  reason  considerable  attention  was  paid  to  the  development  of  a  work¬ 
able  interview  guide.  The  design  of  the  guide  consisted  of  over  15  stages,  involving 
iterative  steps  of  design,  test,  analysis,  and  redesign. 
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A. 1.2  Population 


The  heterogeneity  of  the  population  posed  problems  in  formulating  appropriate 
questions  in  the  interview  guide  and  ensuring  that  the  answers  would  be  comprehensive 
and  unambiguous.  The  population  contains  a  wide  variety  of  scientific  and  engineering 
disciplines;  members  range  from  those  doing  detailed  work  in  their  discipline  to  those 
performing  broad  administrative  and  evaluative  work,  either  in  one  discipline  or  a  mix¬ 
ture  of  disciplines.  It  was  decided  that  the  best  way  to  cope  with  these  problems  was  to 
conduct  an  in-depth  interview  lasting  approximately  two  hours.  An  interview  guide  would 
be  used  and  it  would  contain  ample  space  for  entering  appropriate  narrative  comments. 

A. 1.3  Pilot  Testing 

A  series  of  test  interviews  was  conducted  for  the  purposes  of  testing  the 
interview  guide  and  interview  techniques,  and  as  an  aid  to  interviewer  training.  The 
pilot  testing  was  done  in  two  stages;  the  pre-pilot  stage,  which  consisted  of  18  interviews 
with  technical  personnel  from  the  AUERBACH  Corporation  and  the  Frankford  Arsenal 
in  Philadelphia ;  and  the  pilot  stage,  which  consisted  of  73  interviews  at  DOD  installations 
in  the  Washington  area.  The  Revisions  to  the  interview  guide  and  technique  resulted 
in  changing  from  a  media-oriented  guide  to  one  concentrating  on  the  characteristics  of 
the  information  required  for  a  task,  ;uid  from  an  unstructured  guide  to  a  semi -structured 
guide.  It  should  be  stressed  that  the  purpose  of  the  pilot  interviews  was  not  to  gather 
information,  but  rather  to  provide  feedback  for  improving  all  aspects  of  the  survey 
methodology. 

A.  1.4  Sampling 

Another  important  consideration  was  the  necessity  to  derive  a  representative 
sample  of  the  population.  The  decision  problem  lay  in  the  choice  of  a  sampling  technique, 
i.e. ,  a  stratified  sampling  or  a  simple  random  sampling  technique.  A  stratified  sample 
is  one  in  which  selections  are  made  randomly  from  segmented  groups  of  the  population. 
While  the  stratified  sample  might  have  resulted  in  greater  precision,  the  choice  was  made 
to  use  simple  random  sampling.  The  primary  reason  for  this  choice  was  that  an  insuf¬ 
ficient  amount  of  tested,  a  priori  information  was  available  about  the  pejuilation  to  ensure 
proper  stratification.  The  random  sample  was  checked  by  comparing  the  sample's 
geographic  distribution  against  the  population's  geographic  distribution,  aiui  it  was  found 
to  be  representative. 
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A.  1.5 


Identification  of  Information 


Implicit  in  the  design  of  the  interview  guide  is  the  question  of  how  to  identify 
the  information  required  by  the  user  and  how  to  analyze  the  data  after  it  is  collected. 

A  significant  feature  of  the  study  is  the  development  and  use  of  a  method  for  explicitly 
identifying  those  pieces  of  information  which  were  wanted  and  used  by  the  respondent. 
The  pieces  of  information  used  to  answer  a  particular  question  are  collectively  called 
a  "chunk1'.  A  chunk  is  the  smallest  identifiable  unit  of  task-required  information  which 
would  lose  its  identification  and  meaning  with  respect  to  the  task,  if  segmented  further. 
While  the  concept  of  a  chunk  of  information  does  not  provide  a  quantitative  measure  of 
information,  its  advantage  is  that  it  is  a  workable  and  understandable  unit. 

The  choice  of  information  "measurement,"  however,  posed  a  problem  since 
chunks  are  not  independent  of  one  another  and  since  they  are  by  definition  dependent  on 
the  sampled  tasks.  It  is  therefore  not  possible  to  apply  the  standard  statistical  tests 
in  analyzing  the  data  derived  from  combinations  of  such  questions.  Nevertheless,  it 
was  considered  that  the  definition  and  use  of  the  "chunk"  concept  enabled  a  useful  degree 
of  objective  analysis.  Questions  in  the  interview  guide  winch  do  not  involve  information 
"chunks"  are  still  amenable  to  standard  tests,  and  such  tests  have  been  made. 

A.  1.6  Elimination  of  bias 

The  design  of  the  survey  attempted  to  reduce  and  eliminate  potential  bias  by 
creating  the  proper  environment  in  which  the  interview  was  to  bo  conducted.  Such 
dungs  as  directed  answers  by  superiors  and  concern  on  the  part  of  tho  respondent  that 
his  superior  might  review  his  answers,  were  expected.  To  circumvent  these  problems, 
certain  interviewing  procedures  were  used: 

(1)  Guarantee  of  interviewee  anonymity. 

(2)  No  audience  during  interview. 

(a)  interviewers  would  check  and  cross-check  answers  to 
eliminate  bias  and  test  for  reasonable  answers. 

It  was  found  early  in  the  field  interview  stage  Unit  these  problems  did  not  exist  to  any 
extent.  Most  respondents  had  little  or  no  feeling  about  remaining  anonymous,  and  it 
was  felt  by  the  interviewers  Unit  virtually  no  directed  answers  were  being  ebtait«ed. 
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A.  2  SURVEY  TECHNIQUE 

To  implement  the  objectives  of  the  study,  an  examination  was  made  of  the 
techniques  available  for  collecting  data.  The  following  techniques  were  rejected: 

(1)  Mailed  Questionnaire:  This  approach  was  considered 
not  feasible  because  it  would  have  produced  non-standard, 
probably  biased,  data  which  would  be  difficult  to  interpret. 

(2)  Diary  Technique;  This  technique  would  require  selected 
personnel  to  record  certain  information  in  a  diary  at  the 
end  of  a  given  period  of  time.  The  approach  was  rejected 
for  essentially  the  same  reasons  as  die  mailed  question¬ 
naire. 

(3)  Direct  Observation:  This  technique  would  involve  assigning  an 
observer  to  each  respondent  for  a  period  of  time.  The  purpose 
of  the  observer  would  be  to  observe  and  record  the  information 
the  respondent  acquired  and  used.  The  technique  was  rejected 
because  it  was  impractical  from  a  time  and  cost  consideration. 

The  method  eventually  chosen  was  the  critical  incident  technique  based  on  a 
persona1  interview  with  each  respondent.  This  method  allows  specific  data  to  be  ob¬ 
tained  with  a  minimum  of  bias  and  within  reasonable  time  and  cost  considerations. 

A.  2.1  Development  of  the  Interview  Guide 

The  final  guide  used  in  the  mainstream  of  interview's  evolved  from  a  very 
loosely  structured  instrument  of  informally  phrased  quests  ms  and  minimum  framework 
to  a  semi -structured,  more  formally  organized  interview  guide.  At  no  time,  however, 
did  the  guide  become  a  rigid  questionnaire  of  a  “Yes-No1*  character,  or  of  a  limited 
range  of  responses.  The  interviewer  always  had  tiro  option  of  ignoring  multiple  choices 
or  categorization  if  they  die.  not  fit  the  situation:  the  alternatives  were  narrative  com¬ 
ments  or  outright  omission  of  the  questions. 

(1)  Evolution  to  semi-structured  guide 

The  progression  from  a  very  loosely  structured  to  a 
semi -structured  guide  was  a  natural  one.  The  seroi- 
structured  version  was  possible  only  after  the  pertinent 
questions  and  most  of  their  possible  responses  had  been 
pinpointed  by  the  loosely  structured  pre-pilot  interviews. 

(2)  Discard  Media -Oriented  Guide 

A  second  'ioticeable  trend  during  the  development  of  the 
guide  was  the  shift  away  from  cataloging  the  type  and 


quantity  of  media  used  by  respondents.  Early  versions 
of  the  guide  had,  as  their  primary  objective,  the  analysis 
of  what  journals,  texts,  or  other  type  media  were  instru¬ 
mental  in  shaping  the  respondent's  information  repertoire. 
Gradually,  the  study  team  began  to  realize  that  what  they 
needed  was  a  more  concise  description  of  the  type  of 
technical  information  used,  not  the  name  of  the  media 
from  which  it  was  obtained. 

(3)  Critical  Incident  Technique 

A  third  development  trend  was  the  selection  of  the  critical 
incident  technique.  This  technique  provides  that  an  in¬ 
cident  be  defined,  in  this  case  a  task,  and  a  series  of 
questions  be  developed  to  uncover  the  attributes  and  char¬ 
acteristics  of  tlie  information  the  respondent  used  relative 
to  the  incident.  To  provide  a  basis  of  analysis  of  the 
details  of  the  critical  incident,  further  questions  were 
developed  to  gain  insight  into  the  respondent's  background 
and  into  some  of  Ins  general  information  gathering  habits. 

Each  task  used  as  the  critical  incident  for  the  interview 
met  the  following  criteria: 

(a)  It  was  at  least  eight  hours  in  duration. 

(b)  Some  technical  consideration  was  involved  in  the 
task. 

(o)  It  liad  a  tangible,  clearly  identifiable  output  such 
as  a  technical  report  or  an  oral  briefing. 

(d)  It  was  the  respondent's  most  recently  completed 
task  that  met  the  above  three  criteria. 

The  last  criterion,  that  the  task  must  have  been  completed, 
may  have  resulted  in  a  biased  selection  of  tasks  in  that  a 
disproportionate  number  of  shorter  tasks  arc  likely  to  have 
been  selected. 

<4>  Simplification  of  Recording  Techniques 

A  fourth  development  trend  was  the  attempt  to  simplify 
die  interviewers’  recording  and  reporting  procedures. 

The  largely  narrative  recording  techniques  of  the  early, 
loosely  structured  interviews  required  lengthy  trans¬ 
cribing  and  analysts  by  the  interviewer  after  he  finished 
talking  with  the  respondent.  The  provision  of  additional 
structure  allowed  the  interviewer  to  record  responses 
on  the  guide  itself  by  entering  a  single  letter  or  number 
in  a  block.  *J1u*  simplified  recording  and  reporting 
procedures  reduced  the  average  "after-interview"  cleanup 
lime  from  an  hour  or  more  to  only  a  lew  minutes.  It  very 
likely  increased  the  validity  of  the  returns. 
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(5)  Elimination  of  Current  Awareness  Questions 


Another  development  was  the  elimination  of  specific 
questions  on  the  subject  of  current  awareness.  During 
an  early  analysis  phase  of  the  pilot  study  it  was  recog¬ 
nized  that  unique  problems  existed  in  gathering  accurate 
data  concerning  the  acquisition  and  use  of  current 
awareness  information.  These  problems  are  due  to 
tile  fact  that  current  awareness  information  is  extremely 
encompassing  and  is  gathered  and  used  in  highly  sub¬ 
jective  ways.  Hence,  to  gather  accurate,  comprehensive 
information  concerning  current  awareness,  a  series  of 
close-up,  intensive  studies  would  be  required  where  the 
investigators  would  have  to  "live  with"  the  RDT&E  groups. 
It  was  further  concluded  that  specific  treatment  of  current 
awareness  would  be  inconsistent  with  the  critical  incident 
technique  used  to  identify  task  related  retrospective 
search  patterns.  Therefore,  with  the  concurrence  of  ihe 
contract  monitor,  it  was  agreed  that  the  identification  of 
general  current  awareness  patterns  would  not  be  handled 
within  the  study. 


A.  2. 3  Interviewer  Training 

(1)  Staffing.  The  AUERBACH  team  assigned  to  the  DOD  study 
consisted  of  a  principal  investigator,  project  supervisor, 
project  engineer,  analyst,  training  specialist,  and  inter¬ 
viewers.  In  addition,  consultants  both  within  and  outside 
tlie  organization  were  used.  The  consultants  from  outside 
AUERBACH  were:  Dr.  Robert  Sleight  and  Mr.  Keimeth 
Cook  of  the  Applied  Psychology  Corporation:  Dr.  John 

de  Card  of  die  Statistics  Department  at  the  University  of 
Pennsylvania;  and  l)r.  Herbert  Menzel,  a  researcher  in 
information  use  patterns,  front  Columbia  University. 

(2)  Training.  The  training  program  provided  schooling  tor 
four  classes  of  interviewers:  Class  i.  four  AUERBACH 
interviewers;  Class  11.  four  National  Security  Agency 
interviewers:  Class  HI,  one  additional  AUERBACH  inter¬ 
viewer.  and  Class  IV.  two  backup  interviewers  from 
AUERBACH. 


Hie  iota*  members  of  Class  I  received  an  invaluable  orientation  in  previous 
user  study  research  when  they  collectively  conducted  a  literature  reference  search 
and  compiled  a  very  comprehensive  bibliography  which  comprises  part  ot  the  bibliography 
in  Volume  U.  In  addition  to  receiving  guest  lectures  from  information  specialists  at 
Esso  and  Du  Pont,  the  members  of  Class  I  received  formal  classroom  training  jwiucipaUy 
on  interviewing  techniques;  the  DOI)  organisational  structure:  h«\v  to  make  a  literature 
search;  tiie  philosophy ,  objective,  and  hypotheses  of  the  DOD  study;  meanings  of 


interview  guide  questions  and  how  to  use  the  guide;  interview  recording  and  reporting 
procedures;  and  other  administrative  matters.  The  bulk  of  the  training  time  was  spent 
on  demonstration  interviews  using  the  role-playing  method,  and  in  numerous  "live'' 
practice  interviews  conducted  at  AUERBACH  and  at  DOD  installations  in  the  Washington, 
D.  C.  and  Philadelphia  areas.  The  "live"  practice  interviews  conducted  by  the  regular 
interviewers  were  observed  by  either  the  training  specialist,  the  analyst,  the  project 
engineer,  or  project  supervisor.  The  interviewer  always  received  a  critique  of  his 
performance  from  one  or  more  of  these  observers.  The  interviewer's  performance 
in  completing  the  interview  guide  was  also  thoroughly  analyzed.  Additional  and  signi¬ 
ficant  training  was  received  on-the-job  when  the  interviewers  improved  their  techniques 
during  the  pre-pilot  and  pilot  interviews.  They  also  gained  insight  by  assisting  in  the 
analysis  of  pre-pilot  and  pilot  study  results,  and  providing  assistance  in  the  revision 
of  successive  drafts  of  the  interview  guide.  The  formal  training  concluded  with  a 
field  trip  to  the  Defense  Documentation  Center  at  Cameron  Station,  Virginia. 

Class  II  (the  four  NSA  interviewers)  received  similar  training,  except  that 
it  was  condensed  into  a  two-week  period.  Classes  III  and  IV  (one  and  two  interviewers, 
respectively)  wore  conducted  more  on  a  tutorial  basis  because  of  their  size,  but  again, 
the  training  was  almost  identical  in  content.  The  two-week  com-se  for  Class  II,  which 
formed  the  pattern  for  the  following  classes,  consisted  of  four  days  of  formal  class¬ 
room  training  (including  demonstration  interviews),  and  five  days  of  practice  inter¬ 
viewing  in  the  field,  under  observation  of  the  trainer  and  usually  one  other  student. 

Class  IV,  unlike  its  predecessors,  was  not  able  to  tour  DDC  on  the  last  day. 

Class  I  naturally  did  not  have  the  benefit  of  a  completed  handbook  at  the 
tkogituung  of  its  training.  However,  the  final  drafts  of  the  guide  and  handbook  were 
available  for  all  the  following  classes  to  study  as  they  proceeded  through  their  schooling. 

A . 3  THE  SAMPLING  PLAN 

U  was  decided  early  in  Ute  study  that  a  simple  random  samjjlo  of  between 
1, 200  and  1,300  people  would  he  selected  and  interviewed.  Other  sampling  methods 
may,  no  doubt,  provide  more  precision,  if  more  information  upon  which  to  base  a 
decision  about  the  population  were  available.  There  was,  however,  not  enough  readily 
available  information  about  the  imputation  to  justify  a  different  approach.  The  time 
and  expense  of  collecting  additional  information  were  not  considered  justified  for  an 
initial  study-  It  w<as  therefore  decided  that  a  simple  random  sample  would  be  used 


with  the  understanding  that  it  would  probably  provide,  along  with  other  results  of  the  study, 
information  which  would  be  useful  in  future  studies  and  would  provide  information  leading 
to  sampling  plans  having  more  precision. 

The  population  to  be  studied  was  defined  by  DOD  and  was  furnished  in  the  form 
of  punched  cards  or  on  typed  lists.  Generally,  each  card  contained  the  individual's  name, 
his  serial  number  (if  military),  his  grade,  his  organization,  and  a  number  identifying  his 
professional  field.  In  the  case  of  civil  service  personnel  this  latter  number  is  listed  in  the 
"Handbook  of  Occupational  Groups  and  Series  of  Classes  Established  Under  the  Federal 
Position-Classification  Plan"  prepared  by  the  United  States  Civil  Service  Commission, 
Bureau  of  Programs  and  Standards,  Standards  Division,  and  is  available  from  the  Govern¬ 
ment  Printing  Office.  The  equivalent  information  for  officers  and  enlisted  men  in  the  three 
^’litary  departments  is  available  in  other  publications. 

Names  which  were  furnished  on  typed  lists  were  keypunched,  and  all  cards  were 
then  printed  out  with  a  sequence  number  assigned  to  each  name.  The  total  list  was  com¬ 
prised  of  36, 000  names.  The  next  step  was  to  obtain  random  numbers  upon  which  to  base 
the  sample  selection.  The  Rand  Corporation  publication,  "A  Million  Random  Digits  with 
100,000  Normal  Deviates,"  contains  random  digits  conveniently  arranged  in  five-digit 
numbers;  3,000  random  five-digit  numbers  smaller  than  36,000  were  selected  from  this 
source  and  then  each  number  was  keypunched.  The  cards  were  then  sorted,  placed  into 
numerical  order, and  printed  out.  This  procedure  identified  duplicates,  and  after  rejecting 
them,  about  2,  TOO  sample  cards  remained,  which  are  hereinafter  refer  mi  to  as  the  first™ 
stage  sample. 

At  this  stage  the  question  arose  as  to  how  well,  from  a  geographical  standpoint, 
the  first  -stage  sample  numerically  represented  the  imputation  from  which  it  was  taken. 

The  percentage  of  the  population  located  at  the  various  installations,  or,  in  seme  eases,  a 
generalized  geographical  area  containing  numerous  installations,  was  calculated.  The 
percentage  of  the  sample  at  the  same  installation  or  area  was  also  calculated  and  the  two 
percentages  compared  for  each  location.  In  alt  cases  the  difference  between  the  two 
percentages  was  loss  than  one  percent,  indicating  that  the  sample  could  be  considered  to 
be  representative  of  the  population  from  which  it  was  taken  within  the  parameters  selected. 


After  determination  of  Use  first -stage  sample,  a  card  was  punched  containing  the 
sequence  number  of  the  individual,  his  grade,  job  code,  installation  or  area  code, 


and  military  department.  The  cards  were  then  sorted  by  installation  or  area  code  to 
facilitate  scheduling  and  were  then  printed  out.  Other  sorts  were  printed  out,  such  as 
the  distribution  of  job  codes  within  grades  and  the  distribution  of  grades  within  job  codes, 
for  later  consideration. 

The  second  stage  of  the  sampling  procedure  involved  selecting,  by  random  means 
from  the  first  stage  sample  of  approximately  2, 700  cards,  one  half  of  the  names  at  each 
installation  or  area,  yielding  approximately  the  1, 375  names  required.  Substitute  names 
for  missed  interviews,due  to  people  being  transferred,  retired,  deeeased,or  unavailable 
for  any  reason,  were  selected  randomly  by  installation  from  the  residue  of  the  first-stage 
sample. 


The  study  contract  tutted  that  between  1,200  and  1,850  persons  were  to  be 
sampled.  The  actual  number  of  interviews  performed  was  1,375.  It  is  desirable  to  know 
how  well  this  sample's  characteristics  represent  the  actual  population's  characteristics. 
This  is  found  by  calculating  the  maximum  standard  deviation  for  the  sample  sine  and  using 
the  fact  ti  at  in  95  percent  of  the  items,  th »  sample’s  characteristics  will  differ  from  the 
population's  by  no  more  than  2crmux. 

The  standard  deviation  for  a  simple  random  sample  from  a  population  of  size  N, 
nutl  sample  sue  u  is  given  as 


u  -- 


P  Q 


and 


is  maximized  when  P  Q  -  1/2. 


N,  the  population  size,  is  3b,  000  and  p  equals  1,3 75.  Therefore 

a  3  ^000-1375 

»««x  V  3IUIU0 

1.32  x  UT“. 


lienee,  1*5  percent  of  the  time  the  sample*  $  characteristics  will  differ  from  the 
actual  population's  characteristics  bv  no  more  than  2.  $4  percentage  points  This 

is  considered  to  be  a  conservaUw  upper  bound. 
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THE  SURVEY 


A. 4 

A  4. 1  Method  of  Operation  and  Control  of  the  Main  Survey 

A  preliminary  trip  schedule  based  on  an  expected  rate  of  three  interviews  per 
day  was  established  for  each  interviewer  at  the  s*art  of  the  main  survey,  and  specific  dates 
and  times  were  scheduled  for  eaon  respondent  about  a  month  in  advance  of  each  trip. 

In  scheduling  interviews,  each  interviewer  spent  two  consecutive  weeks  in  the 
field  and  returned  to  the  office  every  other  Friday  for  administrative  purposes  and  for  a 
group  briefing  which  involved  a  two-way  exchange  of  information  between  the  interviewers 
and  the  project  analyst  and  the  project  engineer. 

About  a  month  prior  to  visiting  aa  installation,  tlve  commanding  officer  was 
advised  by  letter  of  the  study  and  of  the  proposed  interviews.  He  was  furnished  a  copy 
of  the  schedule,  and  his  assistance  in  enabling  the  interviewer  “o  carry  out  the  proposed 
interviews  was  solicited.  At  first,  a  letter  was  addressed  to  each  respondent  advising  him 
of  the  study  and  the  proposed  date  and  time  of  the  interview  and  requesting  him  to  advise 
the  prt  ject  engineer  whether  or  not  the  interview  could  be  conducted  Some  difficulty  was 
experienced  duo  to  inadequate  addresses;  consequently,  letters  to  the  respondents  were 
later  forwarded  as  an  enclosure  to  the  letter  sent  to  the  installation  commander,  with  a 
request  that  he  forward  them  to  the  addressees. 

Interviewers  also  experienced  difficulty  and  lost  time  during  th»  first  five  oy  six 
weeks  of  the  main  survey  because  official  notification  of  the  study  had  not  reached  many  of 
the  installation  commanders.  It  is  interesting  to  note  that  during  this  period  about  SU  jmh— 
cent  of  the  scheduled  interviews  were  conducted,  and  during  the  next  few  weeks  this  per¬ 
centage  climbed  to  ‘JO  percent.  Another  important  factor  contributing  to  this  degree  of 
success  was  the  publicity  given  to  the  study  through  semiofficial  news  media. 

Two  news  stories  were  distributed  to  the  services  for  publicity  In  the  appropriate 
media.  The  first  one  was  a  general  story  about  the  purposes  of  ihc  study.  The  second  was 
at  ore  specific,  dealing  with  the  content  of  the  interview  and  identifying  both  the  respondents 
and  the  interviewers  at  specific  installations. 

To  avoid  statistical  bias  by  permitting  the  installation  to  provide  substitutes  for 
unavailable  respondents,  the  schedule  included  a  few  alternates  who  would  normally  He 
interview**!  on  »  suk»ouut*tl  trip.  In  some  cases  the  interviewer  was  able  to  reschedule 
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interviews  after  arriving  at  the  installation,  so  as  to  take  into  consideration  last  minute 
schedule  changes.  When  a  person  was  unavailable  and  unable  to  be  rescheduled  by  the 
interviewer  during  his  visit  at  a  particular  installation,  a  missed-interview  report,  was 
submitted  and  the  person  was  rescheduled  for  a  subsequent  visit  at  that  installation. 

Persons  who  were  transferred,  deceased,  retired,  or  unavailable  for  similar  reasons 
were  considered  to  be  removed  from  the  population  and  substitutes  were  selected  from 
the  first-stage  sample. 

At  installations  where  there  were  more  than  15  persons  to  be  interviewed,  two 
or  more  trips  were  made  over  an  extended  period  of  time  so  that  persons  not  interviewed 
during  earlier  visits  could  be  scheduled  on  subsequent  trips  In  addition,  the  master 
schedule  for  the  main  survey  left  open  the  first  two  weeks  in  October  for  all  interviewers, 
and  the  last  two  weeks  in  September  for  several  interviewers,  in  order  to  reschedule  as 
many  as  possible  of  those  individuals  who  had  been  missed. 

Occasionally  a  respondent  w  '  ,/rite  or  telephone  and  assei't  that  he  should 
not  be  interviewed  because  he  was  not  a  member  of  the  DOD  RDT&E  population.  In  some 
cases,  a  supervisor  took  exception  to  the  selection  of  a  p,.  ••’icular  individual  for  interview 
on  the  grounds  that  he  was  too  new  on  the  job,  or  otherwise  loo  : tiexp  srienced,  and  there¬ 
fore,  he  recommended  a  substitute.  In  such  cases,  it  was  explained  that,  in  order  to 
maintain  statistical  purity,  such  individuals  should  be  interviewed  and  accounted  for  in  the 
analysis  of  the  study.  Table  A-l  shows  some  statistics  in  connection  with  scheduling- 
interviews. 

A ,  4 .  2  Special  Study  of  Parts  Selection  and  Application  Data 

Prior  to  starting  the  main  survey,  a  special  survey  was  performed  for  the  Office 
of  Engineering  Management,  Director  of  Defense  Research  and  Engineering  which  was 
concerned  with  special  needs  for  parts  selection  and  application  data.  The  interview  guide 
developed  for  the  DOD  User  Survey  was  modified  slightly,  and  the  persons  interviewed 
were  selected  from  several  Defense  Department  contractors  in  the  Washington  art  a,  as  well 
as  from  DOD  agencies  also  in  the  Washington  area. 


In  addition  to  the  principal  purpose  of  this  study.it  was  concluded  that  the  general 
interview  technique,  which  was  developed  by  utilizing  the  critical  incident,  would  be  a 
suitable  method  for  solating  the  information  requirements  for  parts  selection  tasks.  The 
critical  incident  approach  consists  of  identifying  :  n  incident,  such  as  a  task,  and  asking  a 
ieries  of  questions  to  uncover  attributes  and  characteristics  of  the  information  respondents 
used  in  the  task. 
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No.  of  Cumulative 


Week 

Ending 

Interview 
Time  in 
Man-Days 

Cum. 

Num¬ 

ber 

Interviews 

Per 

Man-Day 

No.  of 
Inter¬ 
views 

Cum. 

Num¬ 

ber 

Cumulative 
Interviews 
Per  Man-Day 

Percent 
of  Succ. 
Int. 

Percent  of 
Successful 
Interview's 

1  Mav  64’ 

22 

22 

2.2 

48 

48 

2.2 

80 

80 

8 

18 

40 

2.3 

41 

89 

2.2 

78 

79 

15 

21 

61 

2.2 

46 

135 

2.  2 

79 

79 

no 

Lit* 

17 

78 

2.4 

41 

176 

2.3 

79 

79 

29 

21 

99 

2.2 

46 

222 

2.2 

77 

77 

5  June 

25 

124 

2.3 

58 

280 

2.3 

90 

80 

12 

24 

148 

2.7 

64 

344 

2.3 

89 

82 

19 

17 

165 

2.6 

45 

389 

2.4 

90 

84 

26 

25 

190 

2.6 

66 

455 

2.4 

88 

83 

3  July 

22 

212 

2.9 

63 

518 

2.4 

100 

85 

10 

15 

227 

2.7 

40 

558 

2.5 

89 

85 

17 

19 

246 

2.7 

52 

610 

2.5 

91 

86 

24 

20 

266 

2.5 

50 

660 

2.5 

94 

87 

31 

19 

285 

2.5 

48 

708 

2.6 

87 

86 

7  Aug 

24 

309 

2.8 

67 

775 

2.6 

94 

87 

14 

20 

329 

2.8 

55 

830 

2.6 

90 

87 

21 

20 

349 

2.6 

52 

882 

2.6 

88 

87 

28 

21 

370 

2.6 

55 

937 

2.6 

89 

87 

4  Sept 

22 

392 

2.8 

62 

999 

2.6 

89 

87 

11 

14 

403 

2.6 

37 

1036 

2.6 

88 

87 

18 

20 

426 

2.9 

58 

1094 

2.6 

89 

88 

25 

21 

447 

2.5 

52 

1146 

2. 6 

85 

87 

2  Oct 

24 

471 

2.6 

63 

1209 

2.6 

86 

87 

9 

20 

491 

2.4 

48 

1257 

2.6 

94 

88 

16 

18 

509 

2.7 

49 

1306 

2.6 

96 

88 

23 

17 

326 

2.7 

40 

1352 

2.6 

92 

88 

30 

12 

538 

2.  2 

27 

1379 

2.6 

79 

88 

Note:  The  total  number  of  interviews  shown  should  be  adjusted  1379 
for  a  net  deletion  of  four  reports  resulting  from  defective 
reports  and  from  reports  done  prior  to  1  May  1964  _ -4 

Number  of  interview  reports  analyzed  1373 


TABLE  A- 1 .  STATISTICS  OF  INTERVIEW  SCHEDULES 


A  detailed  report  of  this  study  resulted  in  AUERBACH  Technical  Report 
1151-TR-2,  Report  of  Special  Needs  —  Survey  for  Parts  Selection  and  Application  Data, 
April,  1964. 

A.  5  DATA  COMPILATION  AND  ANALYSIS 

A. 5.1  Introduction 

This  part  of  the  study  consisted  of  the  following  steps: 

(1)  Data  preparation:  studying,  editing,  analyzing,  and 
preparing  the  interview  reports  received  from  the  field 
for  computer  processing. 

(2)  Preparation  of  computer  programs  to  perform  data 
processing,  format  generation,  and  selective  search 
of  data. 

(3)  Detailed  analysis  of  the  results  of  the  survey. 

The  first  step,  data  preparation,  was  performed  after  intei'view  guides  were 
filled  in  and  submitted  to  a  centralized  collection  point  at  the  Corporation*  s  facilities  in 
Philadelphia.  The  process  started  when  each  guide  was  analyzed  for  such  factors  as 
completeness  of  clarity,  and  unambiguity  in  terminology.  If  any  items  could  not  be 
resolved  by  the  analyst,  it  was  promptly  returned  to  the  respective  interviewer  for  resol¬ 
ution.  Since  the  guides  wore  submitted  to  the  central  collection  point  at  the  end  of  each 
week,  the  information  never  became  so  old  that  it  could  not  be  accurately  resolved. 
Following  the  acceptance  of  a  guide,  the  information  was  prepared  for  handling  on  both 
the  IBM  1101  and  70-14. 

A .  5 . 2  Computer  Programs.  (Refer  to  Section  V  of  Volume  11  for  a  More  Compre¬ 

hensive  Description.) 

A  versatile  computer  program  capability  was  developed  for  generating  the 
following  outputs: 

(1)  A  simple  distribution  of  the  responses  to  individual 
questions. 

(2)  The  same  as  item  (1)  except  that  certain  classes  of 
responses  to  individual  questions  are  combined  or  pooled 
to  reduce  the  table  size  in  order  to  simplify  the  analysis. 
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(3)  A  frequency  distribution  of  the  responses  to  one  question 
as  distributed  within  the  responses  to  another  question 
for  a  set  of  specified  pairs  of  questions.  These  are  dis¬ 
played  in  matrix  form  with  row  and  column  percentages 
shown.  (Refer  to  Figure  3-1.)  In  addition  each  cell 
contains  a  percentage  based  on  the  row  total  and  a  per¬ 
centage  based  on  the  column  total.  Rounding  errors  are 
shown  for  each  row'  and  for  each  column. 

(4)  The  same  as  item  (3),  but  with  pooled  responses.  The 
computer  program  is  sufficiently  flexible  so  that  either 
the  same  questions,  a  different  set  of  questions,  or  some 
combination  of  questions  may  be  handled  on  a  pooled  basis 
using  the  tabular  arrangement  described  in  item  (3). 

(5)  A  series  of  three-way  tables  giving  the  responses  to  a 
pair  of  questions  as  distributed  against  the  responses 
to  a  third  question. 

((!)  Some  reduced  three-way  distributions.  In  this  case, 

only  certain  preselected  responses  are  printed  out  from 
the  set  of  all  possible  responses  to  the  questions. 

(7)  A  printout  of  all  the  narrative  responses  to  certain 
questions. 


In  addition  to  the  above  printouts,  the  computer  program  has  the  capability  of 
printing  out  ail  but  one  specified  response  to  a  given  question,  either  singly  or  in  combination 
with  other  questions,  in  such  combinations,  one  response  may  be  excluded  from  each  of  the 
questions  concerned.  Another  capability  of  the  program  is  to  print  out  all  serial  numbers 
lor  those  interview  reports  having  soeeified  responses  to  as  many  as  three  specified  ques¬ 
tions.  The  purpose  of  this  feature  is  to  enable  a  researcher  to  identify  and  study  interview 
reports  with  specific  characteristics  or  to  have  a  printout  of  certain  narrative  portions  of 
certain  questions. 


A  selected  portion  of  all  the  tables  is  included  in  Section  VI  oi  Volume  11  of  this 
report.  A  completed  list  of  the  various  questions  which  have  been  printed  out  is  also  con¬ 
tained  in  Section  V! 

A .3.3  Analysis  Procedure 


(1)  Use  of  Chi-Squaml  Tests.  As  mentioned  earlier  in  the  report, 
two-way  tables  of  the  type  used  in  this  study  may  or  may  not 
be  amenable  to  standard  statistical  tests  of  significance.  This 
depends  upon  whether  the  two  questions  being  considered  are 
independent.  In  the  questions  which  are  concerned  with  "chunks" 
of  information,  statistical  testing  was  not  used.  Tins  is  be¬ 
cause,  bv  definition,  they  are  dependent  upon  the  characteristics 
of  the  task. 
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In  selected  cases,  where  the  questions  are  not  concerned 
with  "chunks, "  the  tables  were  subjected  to  standard  chi- 
squared  tests  at  the  95  percent  confidence  level  to  deter¬ 
mine  whether  or  not  a  statistically  significant  relationship 
existed  between  the  classes  of  the  two  questions  involved. 

An  alpha  of  five  percent  (the  probability  of  a  class  I  error  - 
i.  e. ,  rejecting  a  true  hypothesis)  was  selected  and  is 
considered  to  be  adequate. 

(2)  Objective  Analysis.  Analysis  of  the  other  tables  used  the 
following  procedure: 

(a)  All  row  and  column  marginal  percentages  were 
examined. 

(b)  Those  rows  and  columns  that  represented  five 
percent  or  less  of  the  total  population  of  the  table 
were  ignored. 

S  (c)  A  bracket  of  plus  or  minus  50  percent  of  each 

remaining  row  and  column  marginal  percentage 
was  established.  Each  cell  percentage  was  then 
compared  with  its  associated  row  and  column 
marginal  percentage,  and  if  it  were  outside  the 
bracket,  the  cell  was  noted  by  being  circled. 

(d)  If  a  row  or  column  marginal  percentage  were  50 
percent  or  higher,  it  was  arbitrarily  established 
that  the  critical  range  would  bo  20  percent  above 
and  below  that  value.  This  critical  range  or 
bracket  was  then  used  as  in  item  (c) . 

(3)  Subjective  Analysis.  After  the  analytic  procedure  described 
above  had  been  completed,  each  table  was  reexamined  by  the 
analyst.  The  examination  at  this  stage  took  into  consideration 
the  meaning  of  the  two  questions.  Because  of  the  subjective 
nature  of  this  stage  of  the  analysis,  no  specific  criteria  can 
be  described.  Thus,  any  ceil  entry  that  appeared  to  be  out  of 
the  ordinary  because  it  was  higher  or  lower  than  expected 
was  noted.  (This  is  similar  to  the  objective  analysis  dis¬ 
cussed  above,  except  that  the  significance  of  the  question, 
experiences  during  interviews,  etc. ,  were  now  inputs  to  the 
analysis)  If  a  heavy  diagonal  wore  observed,  indicating  a 
strong  dependency  relationship,  or  if,  based  upon  the  subject 
of  the  table,  some  pattern  were  significant  by  its  presence  or 
absence,  this  was  noted.  After  a  consideration  of  these  factors 
comments  were  made  as  to  the  significance  or  meaning  that 
might  be  attached  to  each  tuble. 

(4)  Summary  of  Analysis  Method.  The  analysis  of  the  accumulated 
data  was  done  by  using  two  principal  approaches. 

(a)  The  first  approach  was  based  upon  the  quantitative 
'  data  which  was  produced  by  the  computor  data 
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compilation  programs.  Additional  quantitative 
data  was  produced  by  using  the  chi-squared  test, 
where  appropriate,  plus  the  procedure  described 
under  item  (2) ,  "Objective  Analysis.  "  These  data 
were  used  as  one  type  of  input  to  the  Subjective 
Analysis. 

(b)  The  second  approach  or  method  of  analysis  was 
principally  subjective.  This  was  necessarily  so 
because  of  the  subjective  nature  of  many  of  the 
survey  questions.  All  of  the  coded  and  narrative 
data  was  analyzed  by  subjective  and  intuitive 
methods.  This  was  done  incrementally  over  the 
entire  conduct  of  the  study.  Each  completed  inter¬ 
view  guide  was  read,  oral  feedback  from  the  inter¬ 
views  was  evaluated,  and  the  two  incremental 
computer  printouts  were  examined  in  detail.  The 
incremental  analyses  served  to  establish  a  frame 
of  reference  for  the  analysis  of  the  data  produced 
by  the  third  computer  printout.  The  findings  and 
conclusions  of  this  study  are  essentially  a  synthesis 
and  interpretation  of  those  tables,  data,  observa¬ 
tions  and  evaluations  which  were  found  and  con¬ 
sidered  to  be  the  outstanding  features  of  the 
patterns  identified  of  information  acquisition  and 
use  within  the  DOD  RDT&E  population. 


t 
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This  Appendix  contains  the  survey  data  that  has  been  tabulated  for  each 
survey  question.  The  rationale  and  intent  of  each  question  are  discussed  together  with 
the  validity  and  significance  that  might  be  attached  to  the  resultant  data. 

The  data  obtained  in  response  to  each  question  is  presented  in  tabular  form. 
Each  question  has  a  number  of  answer  categories,  each  of  which  is  identified  by 
letters  or  numbers  found  to  the  left  of  the  category.  In  some  cases,  these  letters  or 
numbers  are  in  a  broken  sequence  indicating  that  the  categories  have  been  pooled.  It 
was  found  useful  to  pool  the  answers  to  several  closely  related  categories  within  a 
question  to  produce  tabulated  data  that  is  more  readable  and  comprehendible.  Should 
a  breakdown  of  the  pooled  categories  or  their  definitions  be  desired,  the  reader  is 
rei erred  to  the  appropriate  question  in  the  Interview  Guide  and  Handbook  contained  in 
Section  IV  of  Volume  H,  to  the  Glossary  of  Terms  in  Volume  I,  and  to  Section  VI  of 
Volume  II,  which  contains  the  unpooled  distributions. 


A  number  of  questions  were  not  applicable  to  particular  respondents  (usually 
in  light  of  the  answer  to  a  previous  question).  For  these  cases,  an  answer  category  of 


"blank"  was  recorded,  which  accounts  for  the  large  number  of  "blanks"  found  in  the  tables. 


Each  table  has  a  "frequency"  column.  The  total  in  tins  column  may  be  any 
of  three  different  totals  depending  upon  the  questions.  That  is,  the  number  1375  re¬ 
fers  to  total  people  or  tasks,  the  number  4G37  refers  to  the  total  number  of  chunks,  and 
the  number  775)0  is  the  total  number  of  the  media  from  which  chunks  were  obtained. 
Each  table  also  has  a  percent  distribution  column  (which  may  not  add  up  to  100  percent 
in  every  case  due  to  rounding). 


Questions  6,  13,  15),  24,  30,  44,  55,  57,  58,  and  GO  provide  narrative  answers 
only.  The  intent  of  these  questions  is  discussed  together  with  relevant  remarks  con¬ 
cerning  their  responses. 


B.  1  DISCUSSION  OF  QUESTIONS  RELATING  TO  RDT&E  PERSONNEL 

B.  1. 1  Discussion  of  Questions  1-8 

Questions  1  through  8  contain  personal  data  about  the  DOD  RDT&E  com¬ 
munity.  The  answers  to  Questions  1  (Military  or  GS  Rating)  and  7  (MOS  or  Job 
Code)  were  verified  from  information  furnished  by  DOD  about  the  total  DOD  RDT&E 
population,  and  spot  checks  indicated  that  the  data  for  these  two  questions  is  reliable. 
Question  6  (Job  Title)  was  not  included  in  the  analysis  because  it  was  found  to  contain 
little  or  no  add  tional  information  beyond  the  responses  to  Question  7  (MOS  or  Job 
Code). 


The  purpose  of  Question  3  (Number  of  TecmacaJ  Personnel  Supervised  at 
Present)  was  to  stratify  the  RDT&E  personnel  according  to  their  supervisory  responsi¬ 
bility,  and  to  gain  insight  into  the  hypothesis  that  those  individuals  who  supervise 
others  have  a  unique  pattern  of  information  acquisition  and  use.  The  data  obtained  for 
Question  3  neither  confirms  nor  negates  this  hypothesis  because  the  answer  categories 
of  this  question  did  not  prove  co  be  mutually  exclusive;  i.  e. ,  in  a  significant  number 
of  cases  it  was  difficult  to  assign  the  answer  obtained  to  a  specific  category. 


QUESTION  1 

MILITARY  OR  GS  RATING 


A. 

GS05, 

GS07, 

00E2,  00E5,  00 EG,  Blank 

Frequency 

53 

Perec 

4 

B. 

0001, 

GS09 

111 

8 

C. 

0002, 

GS11 

210 

15 

D. 

0003, 

GS12 

313 

23 

E. 

0004, 

GSM 

32G 

24 

F. 

0005, 

GS14 

223 

16 

G. 

0000, 

GS15 

113 

8 

U. 

0008, 

GS10, 

0313 

2G 

jj 

TOTAL 

1375 

The  modal  rating  of  the  DOD  RDT&E  community  is  GS13-0004  and  is 
composed  of  85  percent  civilian  and  15  percent  military  personnel. 
The  data  in  tills  table  appeal's  to  bo  normally  distributed. 
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QUESTION  1  (Continued) 


Key  to  Rating 


Air  Force.  Army,  &  Marines 

Navy 

0001 

2nd  Lt. 

Ensign 

0002 

1st  Lt. 

Lt.  (jg) 

0003 

Captain 

Lt. 

0004 

Major 

Lt.  Cmdr. 

0005 

Lt.  Col. 

Cmdr. 

0006 

Col. 

Captain 

0007 

B.  Gen. 

Rear  Adm. 

0008 

M.  Gen. 

Rear  Adm. 

OOE2 

Private 

OOE5 

Sergeant  (Spl  3) 

00E6 

Sergeant  First  Class 

0313 

All  PL313 

*  *  *  *  $ 


QUESTION  2 
YEAR  OF  BIRTH 


10. 

Before  1010 

Frequency 

118 

Percent 

9 

20. 

1011-1020 

207 

22 

30. 

1021-1930 

433 

35 

40. 

1031-1040 

433 

32 

00. 

After  1040 

34 

2 

TOTAL  1375 

The  median  age  of  the  DQD  KDT&E  community  i»  *ni  years.  Fifteen 
percent  are  older  titan  50  years  and  45  percent  are  35  years  of  age 
or  below. 


*  *  *  *  * 


B-3 


QUESTION  3 

NUMBER  OF  TECHNICAL  PERSONNEL  SUPERVISED  AT  PRESENT 


10.  None 

99.  1-10 

Blnk.  Over  10,  Blank 


TOTAL 


*  *  #  *  * 


QUESTION  4 


HIGHEST  DEGREE  AND  FIELD 


No  degree 

Bachelor  in  Aeronautical,  Chemical,  Civil, 
Electrical,  Industrial,  Mechanical,  Metallurgy, 
General,  and  Other  Engineering 

Bachelor  in  Agriculture,  Biology,  Chemistry, 
Mathematics,  Physics,  and  Psychology 

Master  in  Aeronautical,  Chemical,  Civil, 
Electrical,  Industrial,  Mechanical, 

Metallurgy,  General,  and  Other  Engineering 

Master  in  Agriculture,  Biology,  Chemistry 
Mathematics,  Physics,  and  Psychology 

Ph.  D.  in  Aeronautical,  Chemical,  Civil, 
Electrical,  Industrial,  Mechanical, 

Metallurgy,  General,  and  Other  Engineering 

Ph.  D.  in  Agriculture,  Biology,  Chemistry, 
Mathematics,  Physics,  mid  Psychology 

Medicine 

TOTAL 


requenej 


Percent 


Percent 


*  *  #  *  * 


B-4 


s' 


A. 

B. 

C. 
B. 


iUESTION  7 


MOS  OR  JOB  CODE 


Frequency 

Percent 

Biology,  Medical  Officer 

48 

3 

General  Engineering,  Civil,  Electronic, 

Aerospace,  Marine,  Industrial,  and 

Mechanical  Engineering 

697 

51 

General  Physical  Sciences,  Physics,  Chemistry, 
Metallurgy,  and  Meteorology 

301 

22 

Mathematics 

82 

6 

Unknown,  Geography,  Psychology,  Library  and 
Archives,  R&D  Coordinator  (Army), 

Navigator,  and  Photographer 

247 

IB 

TOTAL 

1375 

It  is  of  interest  to  note  that  13  of  the  IB  percent  of  category  E 
represent  tlwse  individuals  who  did  not  know  their  MOS  or  Job 
Code  numbers. 


*  *  *  *  * 


B~5 


A. 

B. 

C. 


Frequency 

Percent 

One  year  and  under 

219 

16 

1-5  years 

555 

40 

Over  5  years 

601 

44 

TOTAL  1375 

The  part  of  this  Question  "this  type  of  work, ''  refers  to  the  MOS  or 
Job  Code  classification  of  Question  7.  It  can  be  seen  that  approximately 
one  in  every  six  persons  has  been  in  his  present  type  of  work  for  one 
year  or  less.  Almost  half  of  the  respondents  reported  they  had  been 
working  in  their  present  type  of  work  for  over  five  years. 

*  *  *  *  * 


B.  1. 2  Discussion  of  Questions  9-11 

These  questions  pertain  to  evaluations  by  the  interviewer  of  the  respondent's 
comments  concerning  his  work  activity.  They  were  developed  with  the  intent  of  pro¬ 
viding  a  profile  of  the  type  of  activities  within  the  DOD  RDT&E  community.  The  cate¬ 
gories  in  Question  11  are  taken  from  the  ASTIA  Distribution  Guide,  dated  January,  1961. 

The  experience  in  the  field  with  Questions  9  and  11  was  excellent  because  the 
categories  were  generally  exhaustive,  unambiguous,  and  mutually  exclusive.  Exper¬ 
ience  with  Question  10  is  considered  very  good;  twwever,  in  evaluating  the  difference 
between  i  eseareh  «uui  exploratory  development,  there  was  a  tendency  to  classify  the 
respondent  into  the  research  category  when  Use  choice  was  not  obvious. 


QUESTION  9 
TYPE  OF  ACTIVITY 


A. 

Detailed  scientific  or  engineering 

Frequency 

786 

Percent 

57 

B. 

Technical  evaluation 

399 

29 

C. 

Technical  administration 

158 

11 

D. 

Other 

32 

2 

TOTAL 

1375 

The  modal  type  of  activity  in  the  DOD  RDT&E  community  is  detailed 
scientific  or  engineering  work.  The  category  "other”  consists  of 
work  involving  no  technical  consideration  whatsoever. 

*  *  *  *  41 


QUESTION  10 
KIND  OF  ACTIVITY 


A. 

B, 

t\ 

l). 

K. 


Research  (and  Exploratory  Development) 

Exploratory  Development,  Advanced  Development, 
Engineering  Development,  and  Operational 
Systems  Development 


671  (510)* 


Reliability  —  Quality  Control 


47 


Percent 
15  (27) 
49  (37) 

3 


ft&D  Support  3  IS  23 

Other,  Blank  129  9 

TOTAL  1375 


This  table  shows  that  about  one-half  o(  the  sampled  DOD  KDT&E 
community  is  performing  a  development  function.  The  category 
"other”  was  used  to  indicate  respondents  obviously  working  In  sev¬ 
eral  categories.  Narrative  comments  associated  with  the  "other'' 
category  indicate  that  at  least  five  percent  of  fhe  nine  percent 
were  principally  engage*!  in  more  than  one  phase  of  development. 

It  should  he  noted  that  the  category  R&D  support  includes  computers 
and  associated  programming  activities. 


If  the  category  Exploratory  Development  is  pooled  with  Research  rather  than 
Advanced  Development,  etc. .  the  pooled  totals  for  categories  A  and  11  would 
be  as  shown  in  the  parentheses. 


QUESTION  11 
FIELD  OF  ACTIVITY 


Frequency 

Percent 

01 

Aircraft  and  Flight  Equipment 

96 

7 

02 

Astronomy,  Geophysics,  and  Geography 

34 

2 

03 

Comical  Warfare  Equipment  and  Materials, 
t  mistry 

53 

4 

05 

Communications,  Electronics 

200 

15 

0G 

Detection 

64 

5 

10 

Fuels  and  Combustion,  Propulsion  Systems 

42 

3 

11 

Ground  Transportation  Equipment,  Transportation 

15 

1 

12 

Guided  Missiles 

117 

9 

14 

Materials  (Non- Metallic),  Metallurgy 

46 

3 

15 

Mathematics 

31 

o 

16 

Medical  Sciences 

82 

6 

22 

Ordnance 

131 

10 

25 

Physics,  Fluid  Mechanics,  Nuclear  Physics 

81 

8 

26 

Production  and  Management 

27 

2 

30 

Research  and  Research  Equipment 

184 

13 

31 

Ships  ami  Marine  Equipment 

48 

3 

34 

Electrical  Equipment,  Installations  and  Construc¬ 
tion,  Navigation,  Nuclear  Propulsion, 

Photo  raplty,  Psychology  aiui  Human 

Engineering,  Quartermaster  Equipment 

77 

6 

36 

Military  Sciences,  Personnel  Training, 
Miscellaneous  Arts  and  Sciences,  Blank 

47 

3 

TOTAL 

1373 

I* exarch  and  ressasvh  equipment  is  a  eftieh-all  category,  ssul 
includes  the  tic!**  ol  computing  and  programming. 
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B.  2 


DISCUSSION  OF  QUESTIONS  RELATING  TO  RDT&E  TASKS 


B.  2. 1  Discussion  of  Questions  12-14 

Questions  12-14  are  intended  to  isolate  and  define  the  tasks  performed  by 
the  respondents.  The  categories  of  Question  12,  field  of  task,  are  identical  to  those 
in  Question  11,  field  of  activity.  Question  13,  relating  to  task  output,  was  an  extension 
of  Question  12,  but  divided  into  subcategories  within  each  discipline.  For  example, 
the  term  antennas  is  a  subcategory  of  electronics,  and  dentistry  is  a  subcategory  of 
medical  sciences.  Question  13  was  not  included  in  the  analysis  because  it  was  found 
that  the  classification  was  too  detailed,  and  thus  segmented  the  disciplines  of  the  tasks 
into  categories  which  were  represented  by  too  few  respondents  to  be  meaningful. 
Question  14  contains  the  same  categories  as  Question  10  except  that  it  is  used  to  de¬ 
fine  the  task. 

The  comments  made  concerning  Questions  10  and  11  also  apply  to  Questions 
14  and  12,  respectively. 


QUESTION  12 
FIELD  OF  TASK 


Frequency 

Percent 

01 

Aircraft  and  Flight  Equipment 

77 

6 

02 

Astronomy,  Geophysics,  and  Geography 

26 

2 

03 

Chemical  Warfare  Equipment  and  Materials, 
Chemistry 

42 

3 

05 

Communications,  Electronics 

203 

15 

06 

Detection 

60 

4 

10 

Fuels  and  Combustion,  Propulsion  System 

44 

3 

11 

Ground  Transportation  Equipment,  Transportation 

9 

1 

12 

Guided  Missiles 

101 

7 

14 

Materials  (Non- Metallic)  Metallurgy 

49 

4 

15 

Mathematics 

27 

2 

16 

Medical  Sciences 

80 

6 

22 

Ordnance 

123 

9 

25 

Physics,  Fluid  Mechanics,  Nuclear  Physics 

85 

6 

26 

Production  and  Management 

30 

2 

3G 

Research  and  Research  Equipment 

215 

16 

31 

Ships  and  Marine  Equipment 

38 

3 

34 

Electrical  Equipment,  Installations  and  Construc¬ 
tion,  Navigation,  Nuclear  Propulsion, 
Photography,  Psychology  and  Human  Engineer¬ 
ing,  Quartermaster  Equipment 

96 

7 

36 

Military  Sciences,  Personnel  Training, 
Miscellaneous  Arts  and  Sciences,  Blank 

70 

5 

TOTAL 

1375 

The  16  percent  figure  for  tasks  in  the  field  of  research  and  research 
equipment  is  misleading  because  it  is  a  catch-all  category  of 
disciplines  and  contains  a  relatively  large  number  of  computing  and 
programming  tasks. 


*  *  *  *  * 
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QUESTION  14 

WHAT  WAS  THE  KIND  OR  LEVEL  OF  THE  MAJOR  OUTPUT  OF  THE  TASK? 


Frequency 

Percent 

A. 

Research  (and  Exploratory  Development) 

176  (329)* 

13  (24)* 

B. 

Exploratory  Development,  Advanced  Development, 
Engineering  Development,  and  Operational 
Systems  Development 

683  (530)* 

50  (39)* 

F. 

Reliability  —  Quality  Control 

62 

5 

G. 

R&D  Support 

357 

26 

H. 

Other,  Blank 

97 

7 

TOTAL 

1375 

*  The  comments  for  Question  10  apply  to  Question  14. 

*  *  *  *  * 

B.  2. 2  Discussion  of  Questions  15  and  16 

The  purpose  of  Questions  15  and  16  was  to  determine  how  tasks  were  as¬ 
signed  or  how  they  originated.  The  intent  of  this  data  was  to  be  used  to  confirm  or 
deny  the  hypothesis  that  .asks  which  are  self- generated  display  a  unique  pattern  of 
information  acquisition  and  use.  The  data  does  not  permit  the  acceptance  or  rejection 
of  this  hypothesis  because  the  categories  of  this  question  did  not  prove  to  be  mutually 
exclusive. 

QUESTION  15 

WAS  THE  TASK  ASSIGNED? 

Frequency  Percent 


A* 

Yes 

709 

52 

B. 

No 

633 

46 

Blnk. 

Blank 

33 

•* 

TOTAL  1375 

*  *  *  *  * 
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QUESTION  16 

IF  THE  TASK  WAS  NOT  ASSIGNED.  HOW  DID  IT  ORIGINATE? 


Frequency 

Percent 

A. 

Self-generated 

507 

37 

B. 

Joint  decision 

107 

8 

C. 

Other 

22 

2 

Blnk. 

Blank 

739 

TOTAL  1375 

54 

The  Blank  category  includes  all  those  people  who  were  not  asked 
this  question  because  their  task  was  assigned. 

*  *  *  *  * 


B.  2. 3  Discussion  of  Questions  17  and  18 

Questions  17  and  IB  attempted  to  determine  the  duration  of  a  task  and  the 
effort  expended  on  the  task  by  the  respondent.  The  data  in  the  table  of  Question  17 
has  been  computed  by  taking  the  recorded  duration  of  the  task  and  multiplying  it  by 
the  midpoint  of  the  range  specified  in  Question  18.  For  example,  if  a  task  were  re¬ 
corded  as  having  taken  two  months  (44  work  days),  and  were  worked  on  one-fourth  of 
the  time  (category  B  of  Question  18),  a  computation  is  made  (1/4  x  44  =  11  days) 
which  puts  the  effective  duration  of  the  task  in  the  second  category  of  Question  17. 

QUESTION  17 


MAN-DAYS  OF  TASK 


Frequency 

Percent 

005. 

1  to  5  days 

735 

53 

022. 

0  to  22  days 

412 

30 

132. 

23  to  132  days 

195 

14 

m. 

over  132  days 

33 

2 

TOTAL  1375 

The  data  in  this  table  is  produced  in  conjunction  with  Question  18  to 
show  the  effective  duration  of  the  tasks.  It  is  noted  that  the  task 
selection  criteria  required  the  sampled  task  to  have  taken  at  least 
one  full  day  of  time. 

***** 


11-12 


ON  THE  AVERAGE.  WHAT  PERCENTAGE  OF  YOUR  OWN  PERSONAL 
TIME  WAS  DEVOTED  TO  THE  TASK? 


Freauency 

Percent 

A. 

20  percent  or  under 

347 

25 

B. 

21  percent  -  40  percent 

277 

20 

C. 

41  percent  -  60  percent 

268 

19 

D. 

61  percent  -  80  percent 

211 

15 

E. 

81  percent  -  100  percent 

243 

18 

Blnk. 

Blank 

TOTAL 

29 

1375 

2 

Over  half  of  the  sampled  tasks  were  performed  utilizing  leas  than  half 
the  person's  time.  Only  18  percent  of  the  tasks  were  worked  on  in 
what  can  be  considered  almost  a  full-time  basis. 


*  *  *  *  # 


B.  2. 4  Discussion  of  Questions  19-23 


Question  19  requires  a  narrative  answer  and  is  intended  to  add  more  infox-- 
mation  to  the  description  of  the  physical  form  of  the  task  output.  Experience  with  this 
question  showed  Ox  at  it  was  a  useful  way  of  aiding  the  identification  of  the  task  output 
and  introducing  the  next  four  questions.  Questions  20  through  23  develop  infox'mation 
about  Uxe  task  output.  They  were  intended  to  determine  whether  or  not  a  relationship 
existed  between  Uie  form  of  Uxe  task  output,  and  the  information  used  to  conclude  Uie 

♦  neb  ru<  f \f\t  qirnh  ti  ti  Anetxi 
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exists. 


JUESTION  20 


WAS  THE  MAJOR  OUTPUT  OF  THE  TASK 


Frequency 

Percent 

A. 

A  finding 

573 

42 

B. 

A  recommendation 

509 

30 

C. 

A  decision 

251 

In 

Blnk. 

Blank 

51 

TOTAL  1375 

4 

TOTAL 


QUESTION  21 

WAS  THE  MAJOR  OUTPUT  OF  THE  TASK 


Frequency 


A.  Oral  286 

B.  Written  935 

C.  Other  124 

Blnk.  Blank  30 

TOTAL  1375 


The  category  "other”  contains  both  oral  and  written  outputs. 

*  *  *  *  * 

QUESTION  22 

WAS  THE  MAJOR  OUTPUT  OF  THE  TASK 

Frequency 


A.  Formal  800 

B.  Informal  552 

Blnk.  Blank  43 

TOTAL  1375 


*  *  *  *  ♦ 

QUESTION  23 

WAS  Till  MAJOR  (Vr;TUT  PIRECTED 

Frequency 


A.  To  or  within  DOD  1154 

B.  Outside  DOD  187 

Blnk.  Blank  34 

TOTAL  1375 


Percent 

21 

68 

9 

2 


Percent 

58 

39 

3 


Percent 

34 

14 


No  attempt  was  made  to  determine  whether  output  directed  to  or 
within  DOD  might  eventually  be  directed  outside  DOD. 


B.3 


DISCUSSION  OF  QUESTIONS  RELATING  TO  CHUNKS 


The  concept  of  a  "chunk"  is  first  introduced  in  the  interview  in  Question  24. 
The  question  is  designed  to  stimulate  a  discussion  so  as  to  identify  the  chunks  of 
information  used  in  the  conduct  of  the  previously  identified  task,  and  to  record  nar¬ 
ratively  a  description  of  each  chunk.  This  question  is  considered  the  pivotal  one  of 
the  entire  interview  because  it  sets  the  stage  for  the  subsequent  series  of  questions 
about  each  chunk. 

Although  the  chunk  concept  is  difficult  to  communicate  properly  because 
most  people  do  not  view  information  this  way,  it  is  felt  that  use  of  this  technique  was 
effective  and  that  the  data  gathered  was  both  exhaustive  and  generally  unbiased. 

The  tables  of  data  which  relate  to  chunks  of  information  (Questions  25  through 
45)  contain  a  relatively  largo  number  of  blank  answers.  These  blank  answers  are  com¬ 
prised  principally  of  those  chunks  of  information  that  were  obtained  from  the  respondent* 
previous  knowledge. 


15.3.1  Discussion  of  Question  23 

Question  23  represents  an  attempt  to  classify  information  into  a  set  of 
mutually  exclusive  categories  for  the  purpose  of  establishing  a  criterion  for  measuring 
the  relative  use  of  different  classes  of  information. 


Definitions  tor  each  category  are  described  in  the  Interview  Guide  Handbook 
which  is  contained  in  Section  IV  of  Volume  II.  in  most  cases  the  interviewer  filled  out 
the  answer  to  tins  question  based  upon  tin?  respondent's  answers  to  Question  24.  This 
procedure  was  followed  to  ensure  uniformity  of  tiic  data. 


Based  upon  a  composite  evaluation  by  Die  interviewers  and  an  evaluation  of 
interviews  observed  by  oilier  members  of  the  study  team,  the  field  experience  and 
die  resulting  data  from  this  question  are  considered  to  be  very  good.  Tins  positive 
evaluation  results  from  a  complete  understanding  of  the  definitions  of  each  class  by 
the  interviewers  and  their  ability  to  implement  them  uniformly. 


13-15 


QUESTION  25 

CLASS  OF  CHUNK 

Frequency 

Percent 

A. 

Concepts 

379 

8 

B. 

Cost  and  Funding 

143 

3 

C. 

Design  Techniques,  Experimental  Processes, 
Production  Processes  and  Procedures, 
Utilization,  and  Test  Processes  and 
Procedures 

940 

20 

E. 

Math  Aids  and  Formulae 

269 

F. 

Performance  and  Characteristics, 
Specifications 

1967 

42 

H. 

Haw  Data 

215 

5 

J. 

Technical  Status 

517 

11 

M. 

Other 

257 

5 

TOTAL 

4687 

The  classes  of  information,  or  chunks,  and  their  frequency  of  use, 
which  were  used  by  the  DOD  RDT&E  community  in  completing  the 
sampled  tasks,  are  shown  in  this  table.  Several  of  the  categories 
have  been  combined,  or  pooled,  since  they  are  closely  related. 

Category  C  is  termed  "how-to-do-it"  information,  and  is  essentially 
procedural  information.  Categories  C  and  F  contain  information 
that  can  be  termed  "Engineering  Data.  ’’ 

***** 

11. 3. 2  Discussion  of  Questions  26  and  27 

Questions  26  and  2?  are  further  attempts  to  qualify  tlm  information  de¬ 
veloped  in  Question  24.  They  are  intended  to  isolate  and  define  the  field  in  which  the 
chunks  apply.  Question  2?  is  not  included  in  the  tabulation  because  it  was  found  that 
the  classification  was  too  detailed  and  thus  segmented  the  disciplines  of  the  chunk 
into  categories  which  were  represented  by  too  few  respondents  to  be  meaningful. 


QUESTION  26 
FIELD  OF  CHUNK 


Frequency 

Percent 

01 

Aircraft  and  Flight  Equipment 

232 

5 

02 

Astronomy,  Geophysics,  and  Geography 

73 

2 

03 

Chemical  Warfare  Equipment  and  Materials, 
Chemistry 

173 

4 

05 

Communication,  Electronics 

762 

16 

06 

Detection 

165 

4 

10 

Fuels  and  Combustion,  Propulsion  System 

131 

3 

11 

Ground  Transportation  Equipment,  Transportation 

34 

1 

12 

Guided  Missiles 

284 

6 

14 

Materials  (Non- Metallic),  Metallurgy 

270 

6 

15 

Mathematics 

197 

4 

16 

Medical  Sciences 

203 

4 

22 

Ordnance 

304 

6 

25 

Physics,  Fluid  Mechanics,  Nuclear  Physics 

324 

7 

26 

Production  and  Management 

156 

3 

30 

Research  and  Research  Equipment 

801 

17 

31 

Ships  and  Marine  Equipment 

92 

2 

34 

Electrical  Equipment,  Installations  and 
Construction,  Navigation,  Nuclear*  Propulsion, 
Photography,  Psychology  and  Human 

Engineering,  and  Quartermaster  Equipment 

319 

7 

36 

Military  Sciences,  Personnel  Training, 
Miscellaneous  Arts,  Blank 

167 

4 

TOTAL 

4687 

♦  *  *  *  * 

B.  3. 3  Discussion  of  Questions  28.  29.  and  30 

Questions  28,  29,  and  30  are  concerned  with  the  means  (hereinafter  referred 
to  as  media)  by  which  the  chunk  of  information  was  conveyed.  In  many  cases  more  titan 
one  media  was  used  to  convey  a  chunk  of  information.  Technical  problems  arose  in 
determining  and  recording  the  actual  number  of  each  media  used  for  a  given  chunk. 


Consequently,  the  data  relating  to  Question  28  is  not  a  reliable  measure  of  the  use 
of  different  media  to  convey  chunks  of  information.  It  does,  however,  provide  an 
indication  of  the  relative  use  of  the  various  media. 

Question  29  was  an  attempt  to  determine  whether  the  media  reported  in 
Question  28  were  normally  used  to  transmit  the  chunks  described  in  Question  24.  The 
question  was  intended  to  determine  whether  selected  media  are  used  to  convey  specific 
kinds  of  information.  The  answers  were  found  to  be  highly  subjective,  and  together 
with  the  limitation  of  the  data  in  Question  28  are  of  marginal  use. 

Question  30  is  a  narrative  answer  question  that  attempted  to  qualify  a  "no" 
answer  to  Question  29.  Based  upon  the  subjectivity  of  Question  29,  these  results  are 
also  of  marginal  use. 

QUESTION  28 
MEDIA 

Frequency  Percent 


A. 

Brochures,  Catalogs,  Standards  and  Codes, 
Drawings,  Schematics,  Parts  Lists,  System 
Specification  Documents  (QMR,  TDP,  etc.) 

872 

11 

B. 

Oral  Contacts  with  Manufacturer,  Oral  Contacts- 
All  Other,  Meetings  and  Symposia 

2276 

29 

C. 

Live  Demonstration,  Physical  Measurement 
or  Experiment 

260 

3 

D. 

Directives,  Handbooks,  Manuals 

579 

8 

K. 

Correspondence,  Memos,  TWX,  Personal  Notes, 
Personal  Logs,  Personal  Files 

634 

8 

N. 

Newsletters,  Other  Mass  Media 

37 

- 

it. 

Reports  and  Proposals 

1289 

17 

s. 

Texts 

446 

6 

T. 

Photographs,  Maps,  Films 

60 

1 

P. 

Pre-Prints,  Reprints,  Journals 

.  379 

3 

V. 

Previous  Knowledge 

630 

11 

VV. 

Computer  Printout 

82 

1 

z. 

Other 

46 

1 

TOTAL 

7790 

The  term  media  refers  to  the  vehicle  by  which  a  chunk  of  informa¬ 
tion  was  conveyed  or  transmitted  to  the  user. 


*  *  *  *  * 
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QUESTION  29 


SI 


■TO 


DO  YOU  HABITUALLY  USE  THESE  MEDIA  OR  PERSONS  TO  OBTAIN 
INFORMATION?  BY  HABITUAL.  I  MEAN.  DO  YOU  NORMALLY  CON¬ 
SULT  THESE  MEDIA  OR  PERSONS  WHEN  CONFRONTED  WITH  THE 
PROBLEM  OF  OBTAINING  THIS  {NAME  OF  INFORMATION  CHUNKS 
KIND  OF  INFORMATION? 


Frequency 

Percent 

A. 

Yes 

3735 

80 

B. 

No 

255 

5 

C. 

Information  Never  Found 

20 

0 

Blnk. 

Blank 

677 

14 

TOTAL 

4687 

*  *  *  *  * 


B.  3. 4  Discussion  of  Question  31 

One  intent  of  Question  31  was  to  determine  and  to  identity  the  physical  ar¬ 
rangement  of  chunks  of  information  as  they  were  received  by  the  respondents.  Another 
intent  of  tiiis  question  was  to  gain  insight  into  the  problems  associated  with  the  packaging 
of  information. 

Experience  in  the  field  and  analysis  of  results  show  that  proper  treatment  of 
this  subject  required  a  more  extensive  series  of  questions  than  those  actually  asked. 

Such  questions  should  consider  ways  in  which  information  is  packaged,  ami  must  provide 
an  exhaustive  (and  lengthy  list  of  packages  with  appropriate  definitions  for  each.  The 
date  in  the  table  for  Questi  n  31  is  quite  limited  because  Ute  question  is  highly  subjective, 
and  the  definitions  for  the  various  categories  were  neither  mutually  exclusive  nor  collec¬ 
tively  exhaustive. 
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QUESTION  31 


AT  THE  TIME  YOU  OBTAINED  THIS  (NAME  OF  INFORMATION 
CHUNK)  WOULD  YOU  HAVE  PREFERRED  IT  PRESENTED  TO  YOU 
IN  ANY  OTHER  MEDIA  OR  IN  ANY  OTHER  PHYSICAL  ARRANGE¬ 
MENT?  (SHOW  LUST  PER  INSTRUCTIONS. ) 


Frequency 

Pore 

A. 

No,  or  no  preference 

2153 

4b 

B. 

Hook,  bulletin,  article,  report 

1027 

22 

C. 

Informal  oral 

329 

7 

D. 

Formal  oral  briefing 

81 

2 

E. 

Live  demonstration 

43 

1 

F, 

Microfilm  or  microfiche 

20 

- 

G. 

Slides  or  motion  pictures 

12 

- 

H. 

Correspondence  and  memos 

107 

■  2 

I  . 

Other  (Specify) 

121 

3 

Bink. 

Blank 

_794 

17 

TOTAL 

4087 

*  *  *  *  * 


B,  3.  3  Discussion  of  Questions  32^3 

Questions  32  uud  .13  attempt  to  define  auid  measure  the  relative  exposure  to 


1W«  »«*V* 


T«uz?  are  vomjwnion  questions  ami  show  the  difference 


between  the  amount  of  exposure  the  respondent  aetindly  obtained  versus  the  amount  of 
exposure  he  wanted.  Definitions  of  the  categories  used  in  these  two  questions  are  con¬ 
tained  in  the  Interview  Guide  Handbook  found  in  Volume  U. 


The  questions  introduce  the  respondent  to  the  concept  that  man>  sources 
may  contain  information  providing'  the  desired  chunk.  The  amount  of  material  is  a 
relative  quantity  defined  by  the  respondent  with  the  assistance  jf  the  interviewer.  The 
field  experience  indicates  that  the  data  is  an  accurate  measure  of  the  exposure  to  in¬ 
formation.  The  interviewers  found  that  this  concept  was  easily  understood  by  the  re¬ 
spondents  and  that  the  categories  Were  exhaustive,  mutually  exclusive,  and  well 
defined. 
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QUESTION  32 


CONCERNING  THE  TOTAL  AMOUNT  OF  MATERIAL  POTENTIALLY 
AVAILABLE  TO  YOU  WHICH  CONTAINS  INFORMATION  ON  THIS 
CHUNK,  HOW  MUCH  OF  IT  WERE  YOU  ACTUALLY  EXPOSED  TO? 


A. 

One  item  of  available  material  which 

Frequency 

1516 

Percent 

32 

B. 

contains  the  information 

A  sampling  of  the  available  material 

1599 

34 

C. 

All  the  available  material 

752 

16 

D. 

Nothing 

36 

1 

Blnk. 

Blank 

784 

17 

TOTAL 

4687 

*  *  *  *  * 


QUESTION  33 

CONCERNING  THE  TOT4L  AMOUNT  OF  MATERIAL  AVAILABLE 
WHICH  POTENTIALLY  CONTAINS  IN  FORMA  TION  ON  THIS  CHUNK, 
HOW  MUCH  OF  IT  DID  YOU  WANT  TO  BE  EXPOSED  TO? 


A. 

One  item  of  material 

Frequency 

1611 

Percent 

34 

B. 

A  sampling  of  the  available  material 

1240 

26 

C. 

All  available  material 

1050 

22 

D. 

Blank 

786 

17 

TOTAL 

4687 

The  17  percent  figure  which  appears  in  many  of  the  tables  concerning 
chunks  is  composed  almost  entirely  of  those  information  chunks  which 
the  respondents  recalled  from  their  own  previous  experience. 


* 


*  * 


*  * 


B.  3. 6  Discussion  of  Question  34 

Question  34  was  asked  only  if  the  answers  to  Question  32  or  33  indicated  an 
exposure  to  a  relatively  large  quantity  of  information.  Consequently,  a  blank  (no 
answer)  was  recorded  for  this  quest- on  in  8]  percent  of  the  cases. 

The  intent  of  Question  34  was  to  determine  whether  a  search  aid  could  have 
helped  the  respondent  select  he  information  he  wanted  to  read  in  detail.  Sample  search 
aids  (a  title  listing  and  an  abstract)  were  shown  to  those  respondents  who  did  not  under¬ 
stand  the  meaning  of  the  term  "search  aid. " 


Because  of  the  large  proportion  of  blanks,  no  general  conclusions  can  be 
drawn  as  to  the  use  of  search  aids.  It  was  presumed  that  those  who  did  not  want  to 
see  a '  rge  quantity  of  information  would  not  need  a  search  aid.  On  the  other  hand,  of 
those  no  needed  a  large  quantity  of  information  and  responded  positively  to  the  question, 
search  aids  would  have  been  useful  or  were  already  being  used  in  68  percent  of  the 
cases. 


QUESTION  34 


SINCE  YOU  WANTED  TO  SEE  A  LOT  OF  MATERIAL.  WOULD  YOU 
HAVE  FOUND  TITLE  LISTINGS  OR  ABSTRACTS  USEFUL  TO  READ 
FIRST  IN  ORDER  TO  HELP  YOU  SELECT  THE  CHUNK  OF  MATERIAL 
TO  READ  IN  DETAIL? 


Percent 


A. 

No,  wanted  to  review  all  the  material  — 

Explain 

263 

6 

B. 

Title  listings,  abstracts,  or  both 

466 

10 

F. 

Already  used  either  or  both 

155 

3 

X. 

Blank 

3803 

81 

TOTAL 

4687 

*  *  *  *  * 
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B.  3.7  Discussion  of  Questions  35  and  36 


Questions  35  and  36  axe  companion  questions  intended  to  determine  the  amount 
of  time  respondents  could  allow  to  get  needed  information,  and  the  time  it  actually  took 
to  get  it.  Responses  to  these  questions  are  considered  very  good  because  they  are  rela¬ 
tively  straightforward  and  are  easily  understood.  In  some  cases  the  respondent's 
answer  indicated  that  he  could  not  have  proceeded  without  the  information.  In  these 
cases  the  question  was  considered  not  applicable, 

QUESTION  35 

FROM  THE  TIME  YOU  REQUESTED  THIS  CHUNK  OR  STARTED  TO 
SEARCH  FOR  IT.  WHAT  WAS  THE  ACTUAL  TIME  IT  TOOK  TO  GET 
IT? 


Frequency 

Percent 

A. 

No,  or  Not  Applicable 

351 

7 

B. 

Under  1  day 

1676 

36 

C. 

Under  1  week 

735 

16 

D. 

Under  1  month,  Under  3  months,  Over  3  months 

895 

19 

G. 

Received  with  task  assignment 

228 

5 

X. 

Blank 

802 

17 

TOTAL 

4687 

*  *  *  *  * 

QUESTION  36 

FROM  THE  TIME  YOU  REQUESTED  THIS  CHUNK  OR  STARTED  TO 

SEARCH  FOR  IT.  WAS  THERE  A  MAXIMUM  TIME  YOU  COULD  HAVE 

ALLOWED  TO  GET  IT? 

Frequency 

Percent 

A. 

No,  or  Not  Applicable 

738 

16 

B. 

Under  1  day 

617 

13 

C. 

Under  1  week 

998 

21 

D. 

Under  1  month,  Under  3  months,  Over  D  months 

1315 

28 

G. 

Received  with  task  assignment 

212 

5 

X. 

Blank 

807 

17 

TOTAL 

468? 

*  *  *  *  ♦ 
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The  modal  time  required  to  obtain  each  information  chunk  was  less  than 
one  day.  The  '’blank1'  answers  are  composed  principally  of  those  information  chunks 
that  were  obtained  from  the  respondent's  previous  knowledge. 

B.  3. 8  Discussion  of  Questions  38  and  39 

Questions  38  and  39  are  companion  questions  designed  to  obtain  responses 
about  the  depth  of  information  wanted  and  received.  The  definitions  of  the  categories 
were  clear  and  the  categories  themselves  were  found  to  be  mutually  exclusive.  The 
data,  however,  should  be  considered  in  light  of  the  fact  that  the  categories  are  inherently 
subjective. 


QUESTION  38 

WHAT  WAS  THE  DEPTH  OF  (NAME  OF  INFORMATION  CHUNK) 


YOU  DECEIVED?  DID  YOU  GET  A: 

* 

Frequency 

Percent 

A.  Once  over  lightly  of  the  subject 

859 

18 

B.  Detailed  analysis 

1994 

43 

C.  Specific  answer 

1002 

21 

D.  Nothing 

42 

1 

Blnk.  Blank 

790 

17 

TOTAL 

4687 

#  *  *  *  * 

QUESTION  39 

AT  THE  TIME  YOU  RECOGNIZED  THE  NEED  FOR  (NAME  OF  IN- 

FORMATION  CHUNK)  WHAT  WAS  THE  DEPTH  OF  THE  INFORMA- 

TION  YOU  WANTED? 

Frequency 

Percent 

A.  Once  over  lightly  of  the  subject 

700 

15 

B.  Detailed  analysis 

2152 

46 

C.  Specific  answer 

1043 

22 

Blnk.  Blank 

792 

17 

TOTAL 

4687 

The  depth  of  the  information  received  and  wanted  is  shown.  Thex'o 
is  no  important  difference  between  the  data  in  the  two  tables. 


*  *  *  *  * 
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1 

B.  4. 9  Discussion  of  Questions  40  and  41 

i 

i  Questions  40  and  41  were  designed  to  inquire  into  the  physical  arrangement 

of  the  information  chunks  as  they  were  both  wanted  and  received.  Information  from 
|  tlies e  questions  is  quite  marginal  for  several  reasons.  First,  the  categories  are  not 

mutually  exclusive;  second,  the  respondents  generally  were  not  concerned  with  the 

) 

)  physical  arrangement  of  the  information,  as  long  as  the  information  was  obtainable. 


QUESTION  40 

HOW  WAS  THIS  CHUNK  OF  INFORMATION  LAID  OUT  WHEN  YOU 
GOT  IT  (NAME  OF  INFORMATION  CHUNK)? 


Frequency  Percent 


A. 

Information  not  obtained 

27 

1 

B. 

Narrative  text;  tables  or  lists; 
graphical-diagrams,  drawings; 
schematics;  flow  charts;  graphs;  maps; 
photos;  graphical  and  text;  graphical  and  lists 

1151 

34 

I  . 

Other  —  Specify 

1706 

36 

X. 

Blank 

1383 

30 

TOTAL 

4687 

***** 

QUESTION  41 

AT  THE  TIME  YOU  REQUESTED  THE  CHUNK  OF  INFORMATION 

HOW  WOULD  YOU  HAVE  LIKED  IT  TO  BE  LAID  OUT? 

Frequency 

Percent 

A. 

Information  not  obtained 

93 

2 

B. 

Narrative  text;  tables  or  lists;  graphical- 
diagrams,  drawings;  schematics; 
flow  charts;  graphs;  maps;  photos; 
graphical  and  text;  graphical  and  lists 

1662 

35 

I . 

Other  combinations  (Explain) 

Other  —  Specify 

1806 

39 

X. 

Blank 

1126 

24 

TOTAL 

4687 

*  *  *  *  * 


i 


The  large  numbers  of  "other"  and  "blank"  in  both  tables  indicate  that  the 
list  of  categories  in  these  questions  was  not  sufficiently  exhaustive.  An  analysis  of 
the  narrative  portions  of  these  questions  indicates  that  there  is  a  wide  variety  of 
physical  formats  both  preferred  and  actually  used. 

B.  3. 10  Discussion  of  Questions  42-45 

Questions  42  through  45  are  concerned  with  the  first  source  or  organiza¬ 
tion  the  respondent  contacted  in  order  to  obtain  each  chunk  of  information  identified 
in  Question  24.  In  the  design  of  these  questions,  it  was  clearly  recognized  that  there 
can  be  a  large  number  of  sources  a  person  eventually  contacts  in  the  pursuit  of  infor¬ 
mation.  However,  the  first  source  is  considered  the  most  significant  because  it  has 
the  inherent  prospects  of  providing  specific  and  immediate  feedback  to  the  interrogator. 

Question  42  attempts  to  define  and  categorize  the  many  first  sources  avail¬ 
able  to  the  respondents.  Information  resulting  from  this  question  is  considered  to  be 
excellent  for  the  following  reasons: 

(1)  The  concept  of  a  first  source  was  not  difficult  to 
transmit  to  the  respondent 

(2)  The  categories  were  well  defined. 

(3)  The  list  of  categories  was  both  exhaustive  and  mutually 
exclusive. 

(4)  The  question  is  fairly  objective. 

Question  43  was  an  attempt  to  find  out  why  the  first  source  identified  in 
Question  42  was  used.  Responses  to  this  question  are  considered  good  because  it  is 
an  easy  question  for  the  respondents  to  understand.  The  data  in  this  table,  however, 
are  considered  quite  marginal  because  it  was  found  that  the  categories  were  not 
mutually  exclusive. 

Question  44  has  only  narrative  answers.  It  was  designed  to  obtain  the 
actual  words  the  respondent  used  to  interrogate  the  first  source  specified  in  Question  42. 
Because  respondents  were  unable  to  recall  the  exact  words  they  had  used,  it  was 
not  possible  to  achieve  this  objective. 
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Question  45  was  designed  to  qualify  and  expand  upon  the  information  ob¬ 
tained  in  Question  42.  The  comments  pertaining  to  Question  42  also  apply  to 
Question  45. 


QUESTION  42 

WHAT  WAS  THE  FIRST  ORGANIZATION  OR  PERSON  YOU  WENT  TO 
IN  ORDER  TO  OBTAIN  THIS  INFORMATION? 


Frequency 

Percent 

A. 

Received  with  task  assignment 

496 

11 

B. 

Supervisor  (if  given  by  Supervisor  after 
assignment),  Assignment  to  Subordinate 

268 

6 

E. 

Consultants  (Outside),  Colleague 

1027 

22 

F. 

Librarian  or  Technical  Researcher,  Library 
(Search  by  Self) 

243 

5 

H. 

Department  Bookcase  or  Files 

608 

13 

I  . 

Own  Collection 

820 

17 

J. 

Information  or  Data  Centers 

19 

- 

K. 

Manufacturer  or  Supplier 

215 

5 

L. 

Blank 

991 

21 

TOTAL 

4687 

*  *  *  *  * 


In  52  percent  of  the  searches  for  information,  the  person  first  used  a  local 
source,  such  as  a  colleague,  his  own  files,  or  local  department  files.  More  than 
half  of  the  21  percent  blank  answers  are  accounted  for  by  information  that  came  from 
a  person's  previous  knowledge.  Libraries  and  information  centers  were  seldom  used 
as  a  first  source  of  information. 
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QUESTION  43 

WHAT  WAS  THE  PRINCIPAL  REASON  YOU  USED  THE  FIRST  SOURCE? 


Frequency 

Percent 

A. 

Received  with  task  assignment 

534 

11 

B. 

Most  authoritative,  only  source  known, 
availability,  previously  found  helpful 

1989 

42 

E. 

Recalled  that  specific  chunk  was  available 
from  this  source 

1119 

24 

G. 

Blank 

1045 

22 

TOTAL 

4687 

***** 

QUESTION  45 

WHAT  DID  YOU  GET  FROM  THIS 

FIRST  SOURCE? 

Frequency 

Percent 

A. 

All  the  information 

1821 

39 

B. 

Part  of  the  information 

1688 

3G 

C. 

Reference  to  further  information 

307 

7 

D. 

Nothing 

96 

2 

Blnk. 

Blank 

785 

17 

TOTAL 

4687 

***** 

The  majority  of  the  17  percent  figures  in  the  "blank"  category  of  Question  45 
is  composed  of  those  information  chunks  that  were  obtained  from  the  respondent's 
previous  knowledge. 

B.  3. 11  Discussion  of  Questions  46-48  and  95 

Questions  46,  47,  48,  and  95  refer  to  the  chunk  of  information  identified  in 
Question  24,  and  are  no  longer  concerned  with  the  first  source. 
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Question  46  was  designed  to  measure  the  essentiality  of  each  chunk  of 
information  to  the  completion  of  the  task  The  data  resulting  from  this  question  is 
considered  somewhat  limited  because  of  the  subjective  nature  of  the  possible  answers. 
A  misunderstanding  concerning  the  implementation  of  this  quustion  tended  to  inflate 
the  "blank"  answers  to  this  question  and  to  Question  47. 

Question  47  was  designed  to  establish  how  chunks  were  actually  used  in 
the  task.  It  was  expected  that  different  types  of  chunks  might  have  different  use 
characteristics.  The  data,  however,  did  not  support  this  hypothesis. 

Question  48  was  designed  to  determine  the  extent  to  which  relevant  and 
existing  information  was  overlooked  during  the  conduct  of  a  task.  The  criterion  that 
was  established  to  permit  a  "yes"  answer  to  this  question  is  explained  in  the  Interview 
Guide  Handbook  in  Volume  IL  Essentially,  it  is  a  fairly  rigid  criterion  which  implies 
that  overlooked  data  either  might,  or  would,  have  changed  the  results  of  the  task. 

Question  95  was  added  after  the  Held  survey  began  in  order  to  qualify  the 
answers  to  Question  48.  It  was  also  intended  to  be  used  to  test  the  hypothesis  that,  as 
the  length  of  time  after  the  completion  of  the  task  increases,  there  is  an  increase  in 
the  incidence  of  "yes"  answers  to  Question  48.  The  data  of  this  question,  when  com¬ 
pared  to  Question  48,  does  not  support  the  hypothesis. 

QUESTION  46 
WAS  THIS  CHUNK 


A, 

Absolutely  essential 

Frequency 

3064 

Percent 

65 

D. 

Could  have  completed  task  without  it 

619 

13 

Dink. 

Blank 

1004 

Zl 

TOTAL 

4687 

Those  information  chunks  that  the  task  could  have  been  completed  without 
did  not  display  unique  or  significant  characteristics. 


QUESTION  47 

WAS  THE  CHUNK  OF  INFORMATION  USED? 


A. 

Directly  in  the  task 

Freauencv 

3142 

Percent 

67 

B. 

As  background  information 

730 

16 

c. 

As  a  lead  to  other  information 

7 

- 

D. 

Not  at  all 

23 

- 

E. 

Other  -  specify 

12 

- 

Blnk, 

Blank 

773 

16 

TOTAL 

*  *  *  *  * 

4667 

QUESTION  48 

AFTER  THE  TASK  WAS  COMPLETED.  DID  YOU  FIND  ANY  INFOR¬ 
MATION  THAT  WAS  AVAILABLE  BUT  UNKNOWN  TO  YOU  AT  THE 
TIME  YOU  WERE  DOING  THE  TASK? 


Frequency 

Percent 

A. 

Yes  (explain) 

173 

13 

B. 

No 

1148 

83 

Blnk. 

Blank 

54 

TOTAL  1375 

4 

The  narrative  comments  associated  with  the  13  percent  "yes"  answers  have 
no  underlying  theme  as  to  why  information  had  been  overlooked. 

■♦**** 
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QUESTION  95 

NUMBER  OF  DAYS  SINCE  THIS  TASK  WAS  COMPLETED? 


Frequency 

Percent 

007 

1  week 

291 

21 

014 

2  weeks 

89 

6 

030 

1  month 

164 

12 

e«o 

2  months 

95 

V 

120 

4  months 

35 

3 

999 

Over  4  months 

17 

1 

Blnk. 

Blank 

684 

50 

TOTAL  1375 

The  large  number  of  ’’blank"  answers  is  due  to  this  question  being  added 
well  after  the  field  interviews  had  begun. 

***** 

B.4  DISCUSSION  OF  QUESTIONS  PERTAINING  TO  INFORMATION  CENTERS 

B.  4. 1  Discussion  of  Questions  49-51 

Questions  49,  SO,  and  51  were  designed  to  obtain  a  measure  of  the  use  of 
TAB  and  DDC  throughout  the  DOD  RDT&E  community  with  the  secondary  intent  of 
identifying  the  reasons  why  DDC  was  not  used. 

QUESTION  49 

HOW  OFTEN  DO  YOU  READ  TAB  ? 


A. 

Almost  every  issue 

Frequency 

280 

Percent 

20 

B. 

Once  every9  2  or  3  months;  no  more  than  once 
every  6  months 

304 

22 

c. 

Never  read  TAB:  does  not  know  of  TAB, 

Blank 

791 

S3 

TOTAL 

1375 

The  20  percent  figure  may  in  reality  be  higher  because  it  was  found  that  in 
many  cases  TAB  was  being  circulated  either  in  parts  or  under  separate  cover  by  local 
libraries.  Thus,  in  some  cases  a  person  may  see  TAB  regularly  but  not  know  it. 

♦  *  *  *  * 

QUESTION  50 

DO  YOU  USE  DDC  (ASTIA)? 


Frequency 

Percent 

A. 

Yes  (skip  to  Question  52) 

640 

47 

3. 

No 

729 

53 

Blnk. 

Blank 

6 

0 

TOTAL 

1375 

***** 

QUESTION  51 

WHY  DO  YOU  NOT  USE  DDC? 

Frequency 

Percent 

A. 

Does  not  know  of  DDC 

286 

21 

U. 

Physical  location;  red  tape;  security;  time; 
poor  previous  results 

43 

3 

F. 

Not  relevant 

169 

12 

H. 

Other  -  Explain 

232 

17 

X. 

Blank 

645 

47 

TOTAL 

1375 

With  the  exception  of  the  first  category  of  Question  SI,  the  remaining  categories 
were  found  to  be  not  mutually  exclusive.  The  narrative  comments  associated  with  this 
question  indicate  that  the  chief  reason  people  do  not  use  DDC  is  because  they  do  not  know 
of  its  existence. 

*  *  *  *  * 
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B.  4. 2  Discussion  of  Questions  52  and  S3 

The  intent  of  Questions  52  and  53  was  to  obtain  a  measure  of  the  use  of 
special  information  and  data  centers  available  to  the  DOD  RDT&E  community.  When  it 
was  found  that  these  centers  were  not  utilized,  an  attempt  was  made  to  identify  the 
reasons  for  the  lack  of  use.  To  aid  in  implementing  Question  52,  each  interviewer  was 
given  a  list  of  33  well-known  and  representative  information  centers  to  be  shown  to  the 
respondent  during  the  course  of  this  question.  The  list  helped  to  define  an  information 
center  and  to  permit  the  respondent  to  recall  any  centers  he  might  have  used. 

QUESTION  52 

DO  YOU  USE  ANY  OF  THE  DOD  SPECIALIZED  INFORMATION 


AND/OR  DATA  CENTERS  SUCH  AS  SHOWN  ON  THIS 

LET? 

Frequency 

Percent 

A. 

Yes  (record  names)  -  skip  to  Question  54 

750 

55 

B. 

No 

604 

44 

Blnk. 

Blank 

21 

TOTAL 

1375 

*  *  *  *  * 

QUESTION  53 

WHY  DON'T  YOU  USE  ANY  CENTERS  OF  THIS  KIND? 

Frequency 

Percent 

A. 

Do  not  know  of  such  centers 

255 

10 

B. 

Physical  location;  red  tape;  security;  time; 
poor  previous  results 

15 

1 

I*. 

Not  relevant 

1S4 

13 

It. 

Other  -  explain 

150 

11 

X. 

Blank 

yn 

50 

TOTAL 

1375 

Thu  narrative  comments  associated  with  this  question  indicate  that  the  chief 
reason  people  do  not  use  information  centers  is  because  they  do  not  know  of  their 
existence. 
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DISCUSSION  OF  ANSWERS  TO  MISCELLANEOUS  QUESTIONS 


B.  5. 1  Discussion  of  Questions  54  and  55 

Question  54  was  designed  to  inquire  into  the  use  of  English  translations  by  the 
DOD  ItDT&E  community,  Resultant  data  is  considered  unbiased  and  accurate  because  the 
respondents  were  able  to  understand  the  question  easily.  This  question  did  not  establish 
the  last  time  a  translation  was  used. 

Question  55  permits  only  narrative  answers  and  is  intended  to  expand  the' ’yes" 
answers  obtained  in  Question  54.  This  additional  data  identifies  the  source  that  pro¬ 
vided  the  translation.  Replies  to  this  question  have  not  been  compiled  from  the  inter¬ 
view  reports  nor  subjected  to  any  analysis  as  part  of  this  study. 

QUESTION  54 

HAVE  YOU  EVER  USED  ENGLISH  TRANSLATIONS  OR  ENGLISH 
ABSTRACTS  OF  FOREIGN  LITERATURE? 

Frequency  Percent 
764  56 

603  44 

6  1 

TOTAL  1375 

B.  5.3  Discussion  of  Questions  56-58 

Question  56  was  used  to  gather  information  concerning  any  difficulties  or 
problems  the  respondent  might  have  encountered  over  the  last  year  in  obtaining  techni¬ 
cal  information.  The  criteria  used  to  allow  a  "‘yes'*  answer  to  be  entered  is  explained 
in  the  Interview  Guide  Handbook.  These  data  can  be  considered  only  as  an  indication 
of  the  DOD  RDT&E  community's  satisfaction  with  their  ability  to  obtain  information. 


A.  Yes 

U.  No 

Ulnk.  Blank 
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QUESTION  56 


DID  YOU  HAVE  ANY  SERIOUS  TROUBLE  OBTAINING  OH  LOCATING 
TECHNICAL  INFORMATION  IN  ORDER  TO  PERFORM  OH  CONCLUDE 

THESE  TASKS? 


Frequency 

Percent 

A. 

Yes 

370 

27 

B. 

No 

921 

67 

Blnk. 

Blank 

84 

6 

TOTAL  1375 

Questions  57  and  58  were  asked  only  if  the  respondent  expressed  a  problem 
in  obtaining  information  (a  Yes  answer  to  Question  56).  These  questions  sought  narrative 
descriptions  of  the  problems  encountered  and  any  proposed  solutions.  A  common  problem 
was  obtaining  information  because  of  security  matters.  The  most  common  suggestion 
from  respondents  (not  necessarily  related  to  the  problems  of  security)  was  that  the  DOD 
publish  and  distribute  a  directory  or  registry  of  the  R&D  effort  currently  underway  through 
out  DOD. 

B.  5.3  Discussion  of  Question  59 

Question  59  was  designed  to  obtain  a  subjective  evaluation  of  the  respondent’s 
requirement  for  information  not  generally  available  through  his  immediate  local  informa¬ 
tion  sources.  The  evaluation  was  made  by  the  interviewer  according  to  the  criteria 
specified  in  the  Interview  Guide  Handbook. 

Question  60  permits  narrative  answers  only.  It  was  intended  to  be  used  by 
the  interviewer  to  add  any  significant  comments  applicable  to  the  answer  given  in 
Question  59. 


QUESTION  59 


HOW  WOULD  YOU  SUMMARIZE  THE  RESPONDENTS  IN  RELATION 
TO  EXTERNAL  TECHNICAL  INFORMATION  NEEDS? 


Freauency 

Percent 

Has  a  very  large  need  for  external  technical 
information 

433 

32 

Has  a  moderate  need  for  external  technical 
information 

597 

43 

Has  insignificant  need  for  external  technical 
information 

330 

24 

Blank 

5 

TOTAL 

1375 

*  *  *  *  * 


B.  5. 4  Discussion  of  Question  93 

The  intent  of  this  question  was  to  gain  some  insight  into  the  feasibility  of 
automating  certain  RDT&E  tasks.  The  answers  to  Question  93  were  determined  by  the 
interviewers  after  the  main  survey  had  been  completed.  These  answers  were  derived 
from  a  series  of  evaluation  questions  which  were  added  during  the  field  survey  plus  a 
reexamination  of  the  completed  interview  guides. 

There  was  a  great  deal  of  difficulty  with  this  question  because  of  the  defini¬ 
tions  of  the  categories  and  the  subjective  nature  of  the  question, 

QUESTION  93 

Categories  1  through  4  represent  an  evaluation  as  to  how  well  the 
specific  task  would  fit  into  a  mon-machine  relationship  in  the  sense 
that  the  task  might  have  been  programmed  and  accomplished  in  some 
automated  fashion.  One  of  the  key  factors  in  making  this  evaluation 
is  to  determine  the  pi  of  ussier  al  know  lodge  or  intellect  needed  to 
perforin  the  task. 
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Frequency  Percent 


1. 

Possibly  mechanize,  low  intellect 

50 

4 

2. 

Difficult  to  mechanize,  some  intellect 

450 

31 

3. 

Very  difficult  to  mechanize,  considerable 
intellect  required 

767 

56 

4. 

Impossible  to  mechanize,  great  intellect 
required 

70 

5 

Blnk. 

Blank 

58 

4 

TOTAL 

1375 

+  *  *  *  * 

B.  5. 5  Discussion  of  Question  94 

Question  94  was  designed  to  obtain  a  method  of  defining  the  general  output 
of  the  tasks.  This  question  represents  an  evaluation  by  the  interviewer  as  to  the 
general  nature  of  the  task,  and  is  based  upon  the  definitions  of  the  categories  used 
in  Question  25.  Question  94  was  added  after  the  field  survey  began. 


QUESTION  94 

EVALUATE  AND  RECORD  THE  OUTPUT  OF  THE  TASK  ACCORDING 
TO  THE  GENERAL  DEFINITIONS  OF  THE  CLASSES  OF  THE  CHUNKS 


Frequency 

Percent 

A. 

Concepts 

46 

3 

B. 

Costs  and  funding;  administrative  action 

137 

10 

c. 

Designs  or  design  techniques 

211 

15 

D 

Experimental  processes  and  procedures 

63 

5 

K. 

Mathematical  aids  and  formulae;  computer 

programs 

94 

7 

F. 

Performance  and  characteristics 

239 

17 

ti. 

Production  processes  and  procedures 

MM 

2 

Uv 

Haw  data 

34 

•A 

L 

Specifications 

66 

5 

J. 

Technical  status 

63 

5 

K. 

Test  processes  and  procedures 

6G 

5 

L. 

Utilization 

23 

2 

M. 

Other;  blank 

75 

5 

N. 

Evaluation 

234 

17 

TOTAL 

1375 

APPENDIX  C .  BIBLIOGRAPHY  OF  USER  STUDIES 


This  bibliography  represents  the  final  compilation  of  references  to  works 
pertinent  to  the  User  Needs  survey  conducted  by  AUERBACH  Corporation  for  the 
Department  of  Defense  under  contract  SD-219.  It  is  an  updated  and  expanded  version 
of  the  Interim  Bibliography  which  was  submitted  as  a  part  of  the  first  monthly  progress 
report  of  the  study.  In  addition,  it  contains  many  entries  from  the  bibliography  prepared 
for  the  National  Science  Foundation  (under  Grant  No.  GN-170)  by  Mr.  Richard  Davis,  of 
Drexel  University;  his  assistance  in  providing  the  survey  team  with  a  draft  copy  of  his 
listing  is  gratefully  acknowledged. 

C  .  1  CATEGORIZATION  OF  USER  STUDIES 

In  order  to  provide  both  effective  access  to  the  nearly  700  entries  and  a  means 
to  examine  the  different  types  of  user  studies,  a  series  of  categories  was  prepared  to 
differentiate  the  studies  into  two  main  groups:  (1)  those  of  interest  to,  and  useful  for, 
only  a  small  segment  of  the  population,  and  (2)  those  applicable  to  more  general  aspects 
of  the  problems  associated  with  the  use  and  creation  of  technical  information.  Each  of 
these  groups  is  further  subdivided  into  subclasses  which  are  discussed  and  defined  in  the 
following  paragraphs  and  winch  pi'ovide  access  to  the  bibliography.  Table  C-l  relates 
these  groups  to  the  bibliography.  Not  all  titles  occur,  because  many  of  the  entries  are 
not  directly  related  to  user  studies,  but  concern  associated  subject  matter:  problem-  in 
Government  information,  report  dissemination  and  control,  interviewing  techniques, 
and  so  on.  Some  titles  may  appear  more  than  once,  for  these  categories  are  not  intended 
as  inflexible  classifications.  They  are,  instead,  general  guides  to  the  contents  of  the 
reports,  which  can  both  illustrato  user  study  methods  and  aid  the  reader  m  selecting 
articles  of  interest  for  him. 

C.  1. 1  The  Limited  InterCvSt  Cateaorv 

The  studies  in  tins  section  were  restricted  in  scope  and  confined  to  a  narrow 
problem  or  investigative  area.  Often  they  were  transitory  in  interest  because  they  were 
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addressed  to  a  specific  local  need.  They  provided  little  insight  into  methodology  required 
for  the  study  of  a  large  and  complex  community  like  the  Department  of  Defense. 

(1)  Studies  of  Specific  Publications 

These  studies  were  concerned  with  single  periodicals 
or  small  and  related  groups  of  periodicals.  The  purpose 
was  to  determine  the  features  of  the  journal  which  were 
well  received,  the  characteristics  users  would  like,  the 
reasons  the  journal  was  used,  the  characteristics  of  the 
reading  audience,  or  other  similar  topics.  Though 
numerous  cited  studies  fall  into  this  group,  they  are  by 
no  means  all  that  exist.  Because  a  great  number  of 
serials,  popular  magazines,  and  other  journals  conduct 
"reader  interest"  polls,  many  were  not  discovered. 

(2)  Journal  Reading  Habits  of  Special  Audiences 

These  studies  are  related  to,  but  quite  dissimilar  from, 
those  in  the  first  group.  Here,  the  focal  point  is  an 
audience  selected  for  certain  special  characteristics, 
e.  g. ,  all  members  of  the  same  discipline,  or  those 
involved  in  the  same  work.  With  this  as  a  starting  point, 
the  studies  were  related  to  journals  read,  or  the  charac¬ 
teristics  of  preferred  journals,  etc. 

(3)  Journal  Reading  Habits  of  Non-Homogeneous  Population 

These  studies  were  much  less  numerous  than  those  in 
groups  which  were  restricted  to  a  specialized  group  of 
readers.  They  were  much  broader  in  concept  and  were 
often  based  on  a  wide  range  of  researchers  in  a  particular 
community.  Often,  for  correlation  purposes,  the  members 
were  divided  into  categories  by  salary,  management  status, 
degree,  and  so  forth. 

(4)  Citation  Studies 

These  studies  examined  the  apparent  media  preference 
deter  mined  through  citations  listed  in  published  material. 

People  who  publish  extensively  are,  to  a  degree,  a  some¬ 
what  select  group.  There  is  noway  of  ascertaining 
exactly  why  the  entry  was  made  in  the  bibliography.  Was 
it  actually  needed  in  the  preparation  o f  »i\e  paper;  was  it 
listed  to  inform  the  reader  of  some  earlier  similar  work 
or  a  useful  reference;  or  was  it  listed  simply  because  the 
author  wanted  to  establish  a  relationship  between  his 
work  ?oui  that  of  others  in  the  field  ? 
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(5)  Facility  Record  Studies 

These  are  similar  to  citation  studies,  but  instead  of 
examining  the  cited  reference  work  in  articles  and  texts, 
they  examine  withdrawal  records  from  libraries, 
librarians'  requests,  or  requests  from  special  infor¬ 
mation  centers.  From  this  the  studies  have  been  applied 
in  two  directions:  to  determine  how  to  improve  or  build 
on  the  information  provided  by  the  facility  itself,  and  to 
determine  characteristics  about  the  literature  used  by 
that  facility's  users. 

(6)  Facility  User  Studies 

These  studies  examined  the  characteristics  or  reading 
habits  of  the  users  of  a  particular  facility.  They  were 
generally  conducted  among  students  in  university 
libraries  and  were  used  to  answer  such  questions  as: 
do  good  students  read  more  extensively,  or  do  the 
reading  habits  of  engineers  differ  from  those  of  English 
majors? 

(7)  Studies  to  Determine  Library  Use 

These  studies  examined  the  use  made  of  a  library.  They 
determined  which  areas  were  used,  and  how;  when 
browsing  was  beneficial;  and  how  students  used  the  card 
catalog. 

(8)  Studies  Soliciting  Opinions  from  Librarians  about  Users 
of  a  Facility 

These  studies  were  wholly  subjective  and  depended  upon 
interviews  or  questionnaires  sent  to  special  librarians  to 
learn  about  the  characteristics  or  "preferences"  of  a 
particular  facility's  users. 

(9)  Studies  Examining  the  Effectiveness  of  a  Particular 
System  or  Piece  of  Hardware 

These  studies  examined  the  efficiency  of  a  certain  pro¬ 
gram  or  piece  of  hardware  in  providing  the  information 
required  by  a  selected  group  of  engineers.  They  were 
very  parochial  in  nature. 

* 

(10)  Mathematical  Models  of  Certain  Aspects  of  the  Storage 
or  Dissemination  offaformation 

These  studies,  primarily  using  the  results  of  studies  in 
group  (5),  developed  mathematical  fools  to  calculate  the 
storage  parameters,  or  use  of  periodicals. 


C-9 


(11)  General  Subjective  Appraisals  01  Problems 


These  were  not  studies  in  the  strictest  sense,  but 
rather  were  evaluations  of  the  information  problem, 
or  some  aspect  or  characteristic  of  users  cu  infor¬ 
mation,  vaguely  based  on  the  published  results  of 
user  studies  performed  by  others.  There  are 
numerous  examples  of  these  works  in  the  literature. 

C.l.  2  The  More  Generally  Directed  Study 

Studies  in  this  category  were  more  broadly  applicable  to  problems  in  the 
acquisition  and  flow  of  information  than  those  in  the  first  group,  and  were  carried  out 
in  most  cases  with  a  degree  of  experimental  control.  As  will  be  discussed  later,  however, 
they  were  limited  by  a  narrowly  or  improperly  drawn  sample,  by  a  faulty  methodology,  or 
by  the  number  of  questions  asked  about  the  use  made  of  the  information.  Even  though 
neither  the  methodology  nor  the  data  from  these  works  could  be  employed  directly  in 
studying  the  information  requirements  of  the  Department  of  Defense  scientific  and  engi¬ 
neering  community  because  of  their  inherent  restrictions,  these  studies  did  aid  in  the 
development  of  the  study  methodology  insofar  as  they  served  as  a  base  to  demonstrate 
where  previous  work  had  failed  or  had  gained  only  limited  success. 

(1)  Media  and  Information  Acquisition  Studies 

Studies  in  this  area  examined  principally  the  media 
or  channel  used  to  acquire  information,  and  were 
more  general  than  any  of  the  studies  identified  in 
Paragraph  C.  1. 1  in  that  they  were  not  restricted 
to  one  journal,  one  library,  one  information  center, 
etc.  Sometimes  the  questions  were  oriented  about 
the  media  or  channel,  and  sometimes  about  the  user. 

In  either  case,  most  studies  in  this  group  looked  at 
one  or  more  of  the  following  topics: 

(a)  Exposure  to  information  channels. 

(b)  Time  spent  reading  different  media  by  a 
small  sample  of  researchers. 

(c)  Media  used  or  preferred  by  r.  selected  category 
of  engineers  or  scientists. 

(d)  Information  channels  used  or  preferred  by  users. 

(e)  Evaluation  of  methods  used  to  acquire  information. 

(f)  The  flow  of  communication  among  a  small  sample 
group. 
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(g)  User  skills  versus  reading  habits. 

(h)  Time  spent  reading  versus  productivity 
or  creativity. 

(i)  Evaluation  of  information  dissemination 
methods. 

(j)  Function  served  by  types  of  media,  e.  g. , 
current  awareness ,  answer  to  a  question. 

(k)  Pseudoquantitative  estimates  of  relative 
value  of  information  gathering  methods. 

(l)  Characteristics  of  preferred  media. 

(2)  Instructions  by  Engineers  to  Aid  in  the  Improvement 
of  Announcement  Services 

These  studies  examined  in  depth  the  desires  of  a  com¬ 
munity  for  certain  types  of  abstracts,  title  listings,  or 
other  forms  of  announcement  services.  They  were 
either  corrective  in  that  they  evaluated  existing 
services,  or  implementative  in  that  they  asked  what 
type  of  service  would  be  desired  if  available. 

(3)  Means  Through  Which  a  Specific  Area  of  Information 
Was  Obtained  and  Examinations  of  Preferences  for 
Certain  Specific  Subject  Areas 

Studies  of  tins  type,  which  were  few  in  number,  chose 
a  special  area  of  information  and  a  specific  discipline- 
related  population  in  order  to  examine  the  means 
through  which  the  information  was  acquired.  One  study, 
for  example,  offered  a  short  list  of  subjects  in  one  field 
and  asked  respondents  which  they  used  most.  These 
studies  have  looked  at  extremely  limited  areas  of  in¬ 
formation  and  narrowly  selected  populations,  usually 
confined  to  but  one  location. 

(4)  Reviews  of  Methodology 

Reports  on  survey  methodology,  though  not  user  studies 
in  themselves,  analyzed  the  present  slate  of  the  art  in 
this  urea  and  served  as  a  valuable  measure  of  the 
methods  develop'd  by  the  survey  team.  Many  major 
reviews  have  been  carried  out,  of  which  the  most 
significant  is  that  of  the  Columbia  University  Applied 
Science  Laboratory  —  see  Item  460  in  the  bibliography. 
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The  IS  different  categories  of  user  studies  discussed  above  represent  a 
general  view  of  the  bulk  of  studies  conducted  to  this  date.  More  detailed  delineation 
of  the  categories  or  characterization  of  their  methodology  was  not  considered  essential 
for  this  report.  If  this  is  desired,  it  is  suggested  that  the  reader  refer  to  Menzel, 
Review  of  Studies  in  the  Flow  of  Information  among  Scientists,  or  Bourne,  Charles  P. , 
Requirements.  Criteria,  and  Measurements  of  Performance  of  Information  Storage 
and  Retrieval  Systems. 

C.  2  REVIEW  OF  PREVIOUS  STUDY  METHODOLOGY 

Several  basic  faults  in  procedures  and  techniques  prevent  the  methodology 
developed  by  earlier  studies  from  being  of  value  in  the  study  of  the  vast  needs  of  the 
Department  of  Defense.  Most  studies  examined  only  a  fragment  population  and 
emphasized  but  few  qualitative  measures  of  the  efficiency  or  characteristics  of  a 
media  or  channel.  The  Columbia  Review  of  the  User  Studies24  considered  this  fault 
as  being  primarily  that  of  developing  limited  descriptive  distributions,  or  simple  cross 
tabulations,  with  few  attempts  at  interpretations  based  on  more  than  one  parameter. 
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APPENDIX  D.  GLOSSARY  OF  TERMS 


Abstract 

Age  of  Information 

Announcement  Journal 

Bibliography 

Boolean  Statement 

Chunk 


Class 

(Of  Information  Chunk) 
Concepts 


A  brief  summary  of  the  contents  of  a  document. 

A  measure  of  up-to-dateness,  in  conventional  units  of 
time,  from  the  present  back  to  the  time  the  subject 
information  was  created.  This  period  of  time  is 
independent  of  the  age  of  the  media  used  to  transmit 
the  information. 

A  secondary  source  journal,  containing  abstracts,  titles, 
indexes,  or  a  combination  of  all  three,  which  is  published 
as  currently  as  possible  with  the  primary  source  in  order 
to  provide  a  current  awareness  service  to  its  readers 
(e.g. ,  Technical  Abstract  Bulletin  (TAB)). 

A  published  list  of  references  or  citations,  each  of  which 
is  relevant  to  a  predetermined  topic. 

A  symbolic  statement  in  the  algebra  of  George  Boole 
which  expresses  logical  connections  between  classes  (in 
the  form  of  AND,  OR,  and  NOT).  The  statement  may  be 
used  to  formulate  an  inquiry  for  the  purpose  of  searching 
a  file. 

A  term  referring  to  a  unit  of  information.  Chunks  are 
discrete  segments  of  the  total  information  required  in  the 
conduct  of  a  task.  A  chunk  is  the  smallest  unit  of  task- 
required  information,  which  would  lose  its  identification 
and  meaning  with  respect  to  the  task  if  segmented  further. 
The  concept  of  chunks  of  information  is  exemplified  by 
considering  chunks  as  the  "pieces"  of  information  required 
for  the  solution  of  a  task.  Thus,  chunks  in  general  then 
can  contain  information  from  a  wide  variety  of  subjects;  they 
can  be  of  various  disciplines,  be  found  in  many  media,  and 
be  either  specific  or  general. 

A  term  used  specifically  in  this  study  to  refer  to  a  list  of 
categories  of  information.  This  list  is  part  of  Questions  25 
and  94  in  the  Interview  Guide. 

Theories,  ideas,  broad  technical  plans,  or  general  relation¬ 
ships.  For  example,  the  plan  for  lunar  orbit  of  manned 
spacecraft  preparing  to  descend  to  the  moon,  or  the  theory 
of  relativity. 


Cost  and  .Funding 


Critical  Incident 


Current  Awareness 


Depth 


Design  Techniques 


Detailed  Scientific  or 
Engineering 


Engineering  Data 


Experimental 
Processes  or 
Procedure 


Exposure 


Field 


The  allocation  or  expenditure  of  money  in  support  of  a 
technical  effort.  For  example,  budget  data  for  the  coming 
fiscal  year,  or  for  the  development  of  a  new  land  mine. 

A  term  used  in  this  report  pertaining  to  a  technique  employed 
in  interviewing.  This  technique  involves  isolating  and  de¬ 
fining  a  mutually  agreed  upon  event  between  the  interviewer 
and  respondent.  In  interviewing,  this  event  usually  forms 
the  basis  for  a  series  of  further  questions. 

A  category  of  information  characterized  by  aiding  a  user 
in  keeping  up-to-date  in  a  particular  field  or  on  a  particular 
subject.  Current  awareness  information  tends  to  assist  in 
maintaining  an  awareness  in  the  state-of-the-art. 

A  term  that,  indicates  the  specificity  or  depth  of  information. 

In  the  context  of  this  study,  it  is  a  scale  for  measuring  in¬ 
formation  chunks  used,  ranging  from  very  specific,  i.e. , 
great  depth  (specific  answer)  to  very  general  depth  (once 
over-lightly  or  overview). 

Detailed  approaches  or  procedures  employed  in  combining 
ideas,  and  the  techniques  of  converting  these  combinations 
into  plans  and  models. 

(Personnel)  who  require  in-depth  technical  expertise  in  a 
particular  area  of  research,  development,  test,  or  evaluation. 
(Capability)  requiring  more  than  just  an  overview  or 
moderately  detailed  knowledge  of  a  scientific  or  engineering 
area. 

A  composite  term  including  the  following  classes  or  subjects 
of  information:  performance  and  characteristics,  specifi¬ 
cations,  production  processes  and  procedures,  test 
processes  and  procedures,  and  utilization. 

A  methodology  used  to  prepare  and  perform  an  investigation. 
Experimental  processes  and  procedures  maybe  generally 
established  ways  of  either  setting  up  or  conducting 
experiments.  An  example  is  the  procedure  for  conducting 
a  wind  tunnel  experiment  to  determine  the  drag  on  a  model 
of  a  new  supersonic  aircraft  configuration. 

A  term  used  in  tins  study  which  is  based  upon  a  relative 
volume  of  information  available  on  the  subject  of  a  given 
chunk  of  information.  Exposure  is  measured  in  terms  of 
tlie  proportion  of  the  total  available  relevant  information 
which  a  user  sees  or  wants  to  be  exposed  to. 

The  33  fields  defined  in  the  ASTIA  Distribution  Guide, 

January,  .1961,  Examples  of  fields  are:  Chemistry, 
Metallurgy,  Propulsion  Systems,  and  Transportation. 
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First  Source 


Information  Center 


Information  Storage 


and  Retrieval 


Math  Aids  and 
Formulae 


Media 


Packaging 


Performance  and 
Characteristics 


Pooled  Data 

Production  Processes 
or  Procedures 

Raw  Data 


RDTfoE 


The  first  person,  organization,  objector  other  external 
source  of  information  consulted  by  the  user  in  a  search 
for  information. 

An  organization  which  acquires,  stores,  indexes,  analyzes, 
and  synthesizes  data  and  information,  usually  on  a  specialized 
subject. 

A  term  generic  to  all  variations  of  the  problems  of  storing, 
locating,  and  selecting  information  of  any  kind,  whether  it 
is  in  graphic  or  digital  form  and  whether  the  desired  output 
is  a  document  or  a  specific  fact. 

Theorems,  equations,  or  empirical  formulas  accepted  as 
standard  information,  and  used  as  tools  in  calculations.  For 
example,  the  formula  for  the  area  of  a  circle,  a  constant  such 
as  7 r,  fixed  tables  such  as  logarithms,  or  Ohm's  law  of  E  =  IR. 

A  composite  term  encompassing  all  vehicles  for  information 
transmission.  Examples  of  media  are:  journals,  oral 
conversations,  drawings,  and  reports. 

The  physical  form  in  which  the  information  was  or  could  be 
conveyed.  Examples  are  4x6  microfiche,  printed  matter, 
on-line  display,  and  so  forth.  Media  is  a  major  aspect  of 
packaging. 

Observed  data  or  qualities  of  an  object  in  terms  of  what  it 
is  or  how  well  it  performs.  Examples  are:  "high  wing 
monoplane,  measured  speed  of  825  knots,"  "...a  linear 
function."  Performance  and  characteristics  indicate  the 
actual  nature  or  capability  of  an  object,  not  the  design 
objectives  of  the  object  (i.e. ,  not  specifications). 

The  combination  of  various  categories  related  to  selected 
questions  found  in  the  Interview  Guide.  Tho  rationale  for 
pooling  categories  is  based  upon  the  experience  of  the 
question  and  the  common  interest  of  the  categories. 

The  method  or  soquence  of  events  involved  in  the  pi'oduction 
or  manufacture  of  an  objeot. 

Unprocessed  and  uncorrected  data  which  arc  the  primary 
record  of  a  scientific  or  technical  measurement  or  event. 

For  example,  a  graphical  record  of  telemetry  data  exactly 
as  it  appeared  when  it  was  radioed  from  a  rocket;  a  high¬ 
speed  photograph  of  the  shock  waves  produced  by  a  projectile. 

Abbreviation  for  Research,  Development,  Test,  and 
Evaluation. 


AUfRtUO* 


Retrospective  Search 

Search  Aid 

Specifications 

S  TAB 

Technical 

Administration 

Technical  Evaluation 

Technical  Status 

Test  Process  or 
Procedure 

Volume 


The  process  of  reviewing  all  present  and  past  information 
in  order  to  select  those  items  which  are  relevant  to  the 
inquiry  which  precipitated  the  search;  emphasis  is  on  the 
quality  and  completeness  of  the  review  and  on  the  presen¬ 
tation  of  the  search  product. 

Search  aids  are  designed  and  produced  for  the  express 
purpose  of  facilitating  the  selection  or  retrieval  of 
relevant  information.  Examples  of  search  aids  are  title 
lists,  abstract  bulletins ,  announcement  journals,  etc. 

Primarily  quantitative  descriptions  of  how  well  an  object 
is  expected  to  perform.  Examples  of  specifications  are: 
"The  proposed  aircraft  must  cruise  at  1000  knots, ", 

"...  the  chassis  is  to  withstand  shock  of  40  g's, ", 

"..  .it  is  required  that  the  computer  be  able  to  operate  in 
an  environment  of  -50  degrees  to  +100  degrees  C." 
Specifications  are  theoretical  expectancies,  not  what  an 
object  is  or  can  do  (i.e. ,  performance  characteristics). 

Abbreviation  for  Technical  Abstract  Bulletin  which  is  an 
announcement  journal  published  by  the  Defense  Documen¬ 
tation  Center  (DDC). 

Personnel  who  spend  90  percent  or  more  of  their  general 
work  effort  in  administrative  duties  rather  than  as  technical 
contributors. 

This  classification  includes  personnel  engaged  principally 
in  monitoring  contractor's  technical  progress  (project 
offices)  or  in  evaluating  proposals.  It  does  not  include 
RDT&E  personnel  who  are  engaged  primarily  in  design 
or  research  but  who  may  occasionally  evaluate  projects, 
plans,  or  proposals. 

The  present  condition  or  state  of  the  art  in  a  scientific 
or  technical  area  or  project.  For  example,  a  quarterly 
progress  report  detailing  the  accomplishments  in  the  develop 
meat  of  a  new  rocket  propellant. 

The  method  or  sequence  of  events  involved  in  duterming  the 
characteristics,  capabilities,  or  limitations  of  an  object 
which  has  been  produced  in  quantity.  For  example,  a 
procedure  for  conducting  desert  trials  of  a  production  model 
of  an  Army  tank,  or  procedures  for  evaluating  durability  of 
common  textiles. 


See  Exposure. 


D-4 


